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Introduction to the Practice Guide on Designing and Delivering 


Career Pathways at Community Colleges 


In today’s labor market, students increasingly 
need greater levels of postsecondary education 
and training to enter or advance in many 
industries.! Meanwhile, employers are calling 
for the education system to better align with 
industry needs, as their demands for a skilled 
workforce continue to rise.” Specifically, 
employers are seeking higher-skilled workers 
who can be flexible, learn new skills on the job, 
and generalize their skills to work across different 
job categories. 


In response to the 


increasingemphasis  S€¢ the Glossary for 
for postsecondary a full list of key terms 
inettaGone ko used in this guide and 


their definitions. These 
terms are underlined 
when first introduced 
in the guide. 


prepare students 

for family-sustaining 
employment in 
high-growth, in- 
demand industries, 
community colleges are employing a number of 
strategies to meet the needs of both job seekers 


and employers. Many of these strategies are 
found within career and technical education 
(CTE) programs, and they are also present in 
broader workforce development activities and 
supports offered by community colleges. 


Career pathways have gained momentum as a key 
strategy for comprehensively and systematically 
enhancing CTE. Career pathways are structured 
around a sequence of connected educational 

and training programs that provide students 

with basic skills and occupational knowledge 

that align with industry needs. When designed 
and delivered well, career pathways combine 
that occupational training with comprehensive 
support services that enable students to advance 
toward higher levels of education and family- 
sustaining careers. The U.S. Department of 
Labor’s Trade Adjustment Assistance Community 
College and Career Training (TAACCCT) grant 
program supported several new and existing 

CTE programs in community colleges to develop 
or expand career pathways. Other examples 


Career Pathway (as defined in the Workforce Innovation and Opportunity Act) 


(7) CAREER PATHWAY.—The term “career pathway” means a combination of rigorous and high-quality 


education, training, and other services that— 


A. aligns with the skill needs of industries in the economy of the State or regional economy involved; 


B. prepares an individual to be successful in any of a full range of secondary or postsecondary 


education options; 


C. includes counseling to support an individual in achieving the individual’s education and career goals; 


D. includes, as appropriate, education offered concurrently with and in the same context as workforce 
preparation activities and training for a specific occupation or occupational cluster; 


E. organizes education, training, and other services to meet the particular needs of an individual in 
a manner that accelerates the educational and career advancement of the individual to the extent 


practicable; 


F. enables an individual to attain a secondary school diploma or its recognized equivalent, and at least 


1 recognized postsecondary credential; and 


G. helps an individual enter or advance within a specific occupation or occupational cluster. 


SOURCE: 29 U.S. Code § 3102 Definitions 
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include: The U.S. Department of Health and 
Human Services invested in career pathways 
through its Pathways for Advancing Careers 
and Education (PACE) and Health Profession 
Opportunity Grants (HPOG) programs. The 
Corporation for National and Community 
Service’s Social Innovation Fund has supported 
the Bridges to Career Opportunities career 
pathways program. The Joyce Foundation’s 
“Shifting Gears” initiative is an example of an 
early investment in planning career pathways. 
A consortium of foundations helped to advance 
career pathways work through the Accelerating 
Opportunity initiative. 


The career pathways approach, as defined under 
the Workforce Innovation and Opportunity Act 
(WIOA) and reinforced under the Strengthening 
Career and Technical Education for the 21st 
Century Act (Perkins V), has become widely 
used in education and workforce development.* 
Because of the emphasis on career pathways 

in CTE programs for the past decade and the 
availability of rigorous evaluation results on 

this topic, this practice guide is focused on 
community college-based CTE interventions 
through the lens of career pathways. 


Developed by the What Works 
Clearinghouse™ (WWC) in conjunction with 
an expert panel, this guide draws upon studies 
of interventions that include one or more of the 
career pathways components defined under the 
WIOA. It focuses on promising interventions 
where there is evidence of improved educational 
or labor market outcomes. This guide 

provides community colleges with five specific 
recommendations for supporting occupational 
skills training through career pathways (see 
Table 1 below). 


Introduction 


Who Might Find This Guide Useful? 


This guide is designed to be used by 
administrators, staff, and faculty at community 
colleges who are responsible for overseeing, 
designing, and/or implementing CTE initiatives.* 
The recommendations in this guide will 

also be useful for policymakers who work 

with community colleges, staff at American 

Job Centers, leaders of community-based 
organizations, and foundations interested in 
supporting workforce development strategies 

at community colleges. This guide may also 

be of interest to employers, labor unions, and 
industry associations that wish to collaborate 
with community colleges to design or operate 
workforce development strategies that are 
responsive to the current and future preparation 
and training needs of the modern workforce. 


Finally, the evidence that supports the 
recommendations in this guide has the potential 
to promote conversations among researchers 
about the availability of rigorous research on best 
practices related to career pathways strategies, 
and where there are research gaps that should 

be addressed. 


Using Evidence to Develop the 
Recommendations 


This practice guide adds to the existing body of 
literature on career pathways by synthesizing 
the evidence from group design studies to make 
five evidence-based recommendations around 
how to design and deliver career pathways 

to support students’ educational and labor 
market success. Each recommendation includes 
examples of career pathways strategies and 
components and how to implement them, advice 
on how to overcome potential obstacles, and a 
summary of the research evidence that supports 
the recommendation. 
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Introduction 


Box 1. Study eligibility criteria Box 2. Levels of evidence 

Eligible research (1) used a comparison group Strong: There is consistent evidence that 
design; (2) included an intervention with a meets WWC standards and indicates that the 
primary focus on developing the technical practices improve student outcomes for a 
skills and knowledge required for specific diverse population of students. 


jobs or fields of work; (3) involved students 
enrolled in community college, or ina 
program sponsored by a community college, 


in the United States; (4) was published in 1999 student outcomes, but there may be 
or later; and (5) reported on one or more ambiguity about whether that improvement 


outcomes in the following domains: (a) credit is the direct result of the practices or whether 
accumulation, (b) postsecondary degree the findings can be replicated with a diverse 
attainment, (c) attainment of a credential population of students. 
(e.g., certification, license, or certificate), Minimal: Evidence may not meet standards 
(d) academic performance, (e) technical skill or may exhibit inconsistencies, but the panel 
proficiency, (fh) short-, medium, or long: determined that the recommendation must 
term employment, or (i-k) short, medium, be included because the intervention is based 
or long-term earnings. on strong theory, is new and has not yet been 
studied, or is difficult to study with a rigorous 


Moderate: There is some evidence meeting 
WWC standards that the practices improve 


The expert panel created a practice guide research design. 
protocol to guide the evidence search and 
review.° Under that protocol, findings from 
eligible studies (see Box 1) that meet evidence 
standards were summarized by trained and 
WWC-certified reviewers for consideration by 


After considering the evidence, the expert panel 
drafted the recommendations and assigned a 
level of evidence to each (see Box 2). 


the panel. The number of studies identified, The five recommendations and the panel’s 
screened, and reviewed is summarized in strength-of-evidence assessment are shown in 
Figure B.1 (see Appendix B). Table 1. 


Table 1. Recommendations and corresponding levels of evidence 


WV) We) im =AYA(0(-Yalexe, 


Practice Recommendation Minimal Moderate Strong 


1. Intentionally design and structure career pathways to enable students 


to further their education, secure a job, and advance in employment. © 


2. Deliver contextualized or integrated basic skills instruction to accelerate 
students’ entry into and successful completion of @ 
career pathways. 


3. Offer flexible instructional delivery schedules and models to improve 
credit accumulation and completion of non-degree credentials along © 
career pathways. 


4. Provide coordinated comprehensive student supports to improve credit 
accumulation and completion of non-degree credentials along career e 
pathways. 


5. Develop and continuously leverage partnerships to prepare students 
and advance their labor market success. 
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Introduction 


Throughout the practice guide, the authors support Recommendation 4 and 11 studies 
refer to the specific interventions that were support Recommendation 5. The number of 
examined in the twenty-one studies that met student participants in the studies ranged from 
WWC standards provide the evidence for this 124 to 42,984. While these studies met WWC 
Practice Guide (see Table 2). Readers can refer standards and informed the evidence-based 
back to this table to get a refresher of some of recommendations in this practice guide, it is 
the intervention details. Recommendation 1 is important for readers to know that the authors 
supported by 12 studies, while Recommendation do not endorse any interventions or programs 
2 is supported by nine studies. Six studies featured in the studies. 


support Recommendation 3, while 15 studies 


Table 2. Overview of Studies Providing Evidence to this Practice Guide 


# of InXXoxeyanlantevareretike) a} 


Intervention Study Students | 1 A | 8 | 4 | 
Accelerating Opportunity Anderson et al. (2017) 42,984 e e e 
Additional source: Anderson et al. 
(2015) 
ACED Grant at Salt Lake Community Bragg et al. (2018) 1,448 e 
College 
Advancing Careers and Training (ACT) Price et al. (2018) 3,376] e e 
for Healthcare in Wisconsin t ae i 
Career Focused Learning Communities Visher & Teres (2011) | 917 e 
Carreras en Salud Martinson et al. (2018) 800 e lve) Ser |e 
Grand Rapids Community College Martinson et al. (2016) ] 274 e e 
Pathways to Prosperity Grand Rapids Community College al 
Integrated Basic Education and Skills Glosser et al. (2018) 631 e e e 
iieelining Pregtain( BES Ny) Modicamore et al. (2017) 5468) 6 |< is 
Accelerating Connections to 
Employment I | 
Kern Community College Clean Energy Martinson et al. (2016) | 816 e e e 
Center Kern Community College District | 
Madison Area Technical College Cook et al. (2018) 500 e e 
Patient Care Pathway | | Zs 
Making the Future: The Wisconsin Price et al. (2016) | 6,300 e e e 
Strategy | 
New Visions Self-Sufficiency and Fein & Beecroft (2006) 1,043 e |e 
Lifelong Learning Project 
North Central Texas College Health Martinson et al. (2016) 984 e 
Matrix Grant scholarship program North Central Texas College 
Northeast Resiliency Consortium | Price et al. (2017) 1,396 | e 
Nursing STRIPES Curl et al. (2016) | 424, | |e | | 
Pathways to Healthcare | Gardiner et al. (2017) 1,217 | e e e | e | 
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nXXexeyanlantsyarersticed a 
He) 
Intervention Study Students | 1 | 2 | 3 | 4 | i) 


Project QUEST Roder & Elliott (2018) 410 e 


Additional sources: 
Roder & Elliott (2019) 
Elliott & Roder (2017) 


ShaleNet Dunham et al. (2016) 353) 1s e 


VIDA Program Rolston, Copson, & Gardiner 958 e 
(2017) 
Workforce Training Academy (WTA) Hamadyk & Zeidenberg (2018) 943 e e e e e 
Connect 
Year Up Fein & Hamadyk (2018) 2,544 e e 
Overarching Themes flexible course scheduling and instructional 


delivery formats (Recommendation 3), and 
assessing student needs for different types 
of academic and non-academic supports 
(Recommendation 4). 


In developing the practice guide, the expert 

panel identified five overarching themes that hold 
relevance across multiple recommendations. 
These themes represent important issues or topics 
that the expert panel believes are important to * The importance of aligning career 


consider when developing and implementing 
career pathways. The five themes are: 


¢ The importance of understanding the 
student population. When planning or 
updating a career pathway, administrators, 
faculty, and staff should carefully consider 
the student population they both target and 
serve. It is important for community colleges 
to understand their student population 
and what students need in order to be 
successful when pursuing a career pathway. 
By understanding the student population, 
community colleges and their partners are 
better able to successfully design and deliver 
career pathways that provide appropriate 
forms of instruction and support services to 
accommodate students with different needs 
and capabilities. As described more fully 
under the individual recommendations, 
understanding the student population 
facilitates connecting students to pathways 
that align with their academic interests 
and career goals (Recommendation 1), 
determining appropriate teaching 
strategies for basic skills training 
(Recommendation 2), identifying needs for 


pathways to industry needs. The panel 
believes that aligning career pathways to 
industry needs is another important theme that 
cuts across several of the recommendations. 
The skill sets and levels of preparation required 
for successful employment vary considerably 
from one industry to another, and even from 
job to job within an industry. Therefore, 

career pathways that prepare students for 
manufacturing jobs will differ from those 
designed to prepare students for healthcare 
jobs, for example. When considering the 
recommendations in this guide, community 
colleges and their partners should determine 
how best to apply the recommendations in 
their efforts to prepare students to thrive in 

the evolving industries students are pursuing. 
The panel provides guidance on aligning 
career pathways with industry needs as 

part of designing career pathways content 

and curriculum, structure, and credentials 
(Recommendation 1), as well as sustaining 
and leveraging industry partnerships to provide 
hands-on learning opportunities to prepare 
students and advance their labor market 
success (Recommendation 5). 
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¢ The need for faculty professional 


development. The panel recognizes that 
several of the recommendations in the present 
practice guide may require professional 
development for career pathways faculty. 
High-quality instruction is a key component 
of effective career pathways. Most of the 
studies supporting the recommendations 

in this practice guide were of interventions 
that targeted and served older students.°® 

For this reason, the panel believes all faculty 
can benefit from pedagogical training that 
specifically addresses adult learning theory 
and pedagogies for effectively teaching adult 
learners, thereby enabling them to meet 
students’ diverse learning needs. The panel 
provides more guidance on professional 
development for faculty as part of supporting 
occupational and basic skills instructors 

that are team teaching and delivering 
integrated basic skills training 
(Recommendation 2), instructors designing 
and delivering flexible instructional delivery 
models (Recommendation 3), as well as 
faculty and staff coordinating and 
collaborating around delivery of student 
supports (Recommendation 4). 


Many CTE programs hire industry experts 

to teach occupational courses because they 
have technical content knowledge. Yet, many 
of these experts lack teaching experience 

or the necessary credentials to teach at the 
college level. As such, additional funding and 
training may be necessary to support faculty 
development, especially when such faculty 
are first transitioning out of industry and into 
teaching. The expert panel believes training 
of faculty should cover practical tips for 
generating syllabi, delivering instruction, and 
creating assessments. Further, administrators 
should also consider providing peer 

mentors for new faculty and encouraging 
professional communities of practice. 
Faculty in occupational programs who do 
not have an advanced degree can pursue 

one through degree programs for workforce 


Introduction 


educators. For example, South Seattle College 
offers a Professional Technical Education & 
Instructional Design program that prepares its 
participants for technical teaching positions at 
community and technical colleges.’ 


The importance of adequate time and 
resources. The panel believes that community 
college faculty and staff need ample time 

and resources to collaborate as they design 
and deliver career pathways instruction. 
Faculty should be involved in designing 
career pathways to ensure that one program 
adequately prepares students for the next 
program along the pathway. Faculty and staff 
must also be afforded opportunities to identify 
student needs and coordinate supports that 
will meet those needs. The panel brings 
particular attention to the importance of time 
and resources in relation to curriculum 
redesign and development 
(Recommendation 1), collaborative 
planning and delivery of team teaching 

and contextualized basic skills training 
(Recommendation 2), design and delivery 
of flexible instructional models and formats 
(Recommendation 3), coordination 

and collaboration between staff and 

faculty around student support services 
(Recommendation 4), and establishment of 
sustained employment focused partnerships 
(Recommendation 5). 


The panel recognizes that funding for career 
pathways is not equal at all community 
colleges, and that recent national events 
requiring increased remote teaching and 
learning may have exacerbated inequalities 
across community colleges even further. 


Opportunities for more rigorous research 
on the effectiveness of career pathways. 
This practice guide focuses on components of 
career pathways that are implemented at—or 
in partnership with—community colleges. 
Much of the available research on career 
pathways at the community college level 
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focuses on interventions that result in non- 
degree credentials for students initially lacking 
the math, reading, or writing skills required to 
successfully complete college-level work. To 
date, few rigorous studies report on medium- 
and long-term educational or labor market 
outcomes. The panel believes that more 
research is needed on the full spectrum of 
career pathways interventions, including 
those that extend beyond the community 
college to include long-term outcomes. 

The panel provides guidance on 

examining the delivery and outcomes 

of contextualized and integrated basic 

skills training (Recommendation 2), 
monitoring students’ needs and use of 
flexible instructional scheduling and delivery 
models (Recommendation 3), evaluating 
students’ need for and use of support services 
(Recommendation 4), and assessing 
benefits and challenges of partnerships 

with local employers and industry partners 
(Recommendation 5). 


The panel recognizes that community colleges 
vary in their capacity to collect and use data to 
understand student progress and outcomes in 
career pathways. CTE researchers can support 
community colleges to better understand the 
types of data they should collect to examine 
their students’ education and employment 
outcomes throughout an entire career 
pathway. When feasible, the expert panel 
believes colleges should use administrative 
and program implementation data to assess 
student demand for certain programs and to 
understand rates at which students persist, 
complete, and transition from career pathways 
programs to employment. 


How to Use This Practice Guide 


Each recommendation focuses on a particular 
aspect of designing and delivering career 
pathways at community colleges. For each of the 
five recommendations in this guide, we include 
the following: 


Introduction 


¢ The recommendation: Details about the 
recommended practice, example career 
pathways strategies, level of evidence, and 
a description of how the recommended 
practice supports student outcomes. 
Appendix C contains a detailed rationale for 
the level-of-evidence category assigned to 
the recommendation (strong, moderate, or 
minimal), and information on the individual 
studies that support it. Each recommendation 
also includes a “Highlight From the Field” 
that provides additional implementation detail 
from one of the supporting studies. All figures 
and mentions of specific community colleges 
in this guide are offered as examples only and 
should not be read as endorsements of specific 
products or approaches. 


¢ Howto Carry Out the Recommendation: 
Guidance on how to implement the 
recommended practice. This guidance is 
informed by the studies that support the 
recommendation as well as the expert panel’s 
expertise and knowledge of resources and 
strategies available to help implement the 
recommendation. 


* Potential Obstacles and the Panel’s Advice: 
Advice from the expert panel about design and 
implementation challenges and how readers 
should overcome them. 


¢ Tools and Resources: Examples related 
to the guidance for how to carry out the 
recommendation. 


The guidance on how to carry out each 
recommendation is presented in a sequence. 
However, not every step will be appropriate 

to every audience, nor is every step required 

in order to implement a recommendation 
successfully. Users of this guide are encouraged 
to use the advice provided here in ways that fit 
best into the varied contexts in which they work. 
Though this guide does not provide step-by-step 
instructions for implementation, readers will find 
resources mentioned throughout providing more 
details about how to apply particular practices. 
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Recommendation 1: Intentionally design and structure career 


pathways to enable students to further their education, secure a job, 
and advance in employment. 


Career pathways can be designed and 
implemented in many different ways. Their 
structure and content will vary depending on 
the occupations and industries targeted; the 
skills and knowledge required for employment 
in these occupations and industries; and the 
education and training programs needed to 
meet these requirements. Across occupations 
and industries, what career pathways have 

in common is that they comprise a series of 
connected education and training opportunities 
that enable students to secure a job or further 
their education or employment. 


When designed and implemented well, career 
pathways offer students a clear blueprint for 

educational and employment advancement. A 
basic career pathway model includes multiple 


Figure 1.1. Basic career pathway model 


entry points to facilitate access to training in 
occupational skills and knowledge valued by 
employers and stackable credentials that match 
labor market demand (Figure 1.1). Multiple exit 
points aligned with employment opportunities 
offer participants flexibility to access employment 
at different stages of the career ladder. Individuals 
may enter and exit career pathways at multiple 
points as they advance in their careers. 


The flexibility of pathways enables students to 
start at a level commensurate with their skills. 
Therefore, not all students will start at the first 
level in the model. Bridge programs, which are 
typically preparatory courses offered to students 
in need of basic skills, such as math, reading, or 
writing, are not always offered at the beginning of 
career pathways. 


Basic Career Pathway Model 
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2 
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Exit Point 


Exit Point 


Exit Point 


Occupational Knowledge and Skills 


Source: Adapted from Alliance for Quality Career Pathways (2014), Fein (2012), and Wachen et al., (2010). 
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Clearly defined career pathways allow students 
to see how their progression through individual 
educational and occupational training programs 
can lead to specific employment opportunities— 
providing students with an encouraging 
roadmap. Each step of the career pathway is 
designed for students to successfully progress 
to the next level of education and employment. 
Student access and progression along the career 
pathway can be further motivated by simplified 
enrollment and intake procedures and smooth 


transition points between training and education. 


WWC staff and the expert panel assigned a 
moderate level of evidence to Recommendation 1, 
based on 12 studies of interventions with career 
pathways components. Eight of the studies® 
meet WWC group design standards without 
reservations, and the other four studies® meet 
WWC group design standards with reservations. 


Recommendation 1 


In all 12 of the studies used to support 
Recommendation 1, career pathways are a 
major component of the tested intervention. See 
Appendix C for a detailed rationale for the level 
of evidence assignment for Recommendation 1. 


This section describes strategies, examples, 
and tools that can support the intentional 
design and delivery of career pathways that 
enable students to enter and advance in 
employment and further their education. 
These career pathways can be developed by 
community college administrators, faculty, 
and staff, in partnership with employers and 
industry stakeholders. All figures and mentions 
of specific community colleges in 
Recommendation 1 are offered as examples 
only and should not be read as endorsements 
of specific products or approaches. 


Table 3. Results of Meta-analysis for Recommendation 1 


Statistically 


Yefaliicer-laiar-lale| 
positive 


Outcome Domain 


Statistically 
Yio Tatnicer-talar-lale| 


Indeterminate rat=Xe FL ahy-) 


Credit accumulation @ 


Postsecondary degree 


attainment 
Industry-recognized credential e 
attainment 
Short-term employment e 
Short-term earnings @ 


Note: Across the 12 studies, there were findings in five outcome domains even though all 11 outcome domains (see Box 1) 
were relevant for this recommendation. None of the 12 studies included findings that meet WWC standards in the following 
outcome domains: (1) academic performance; (2) technical skill proficiency; (3) medium-term employment; (4) long-term 
employment; (5) medium-term earnings; or (6) long-term earnings. 
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How to Carry Out the 
Recommendation 


The guidance below is informed by the studies 
that support the recommendation as well as 
the expert panel’s knowledge of strategies 

and resources available to help implement the 
recommendation. 


1. Align career pathways to industry needs. 


When developing career pathways, 
community colleges should assess and 
identify regional industry needs to ensure 
the pathways will provide students with 

the occupational skills and knowledge 

that employers demand.” The expert 

panel suggests community colleges should 
focus on developing career pathways that 
effectively lead to higher credentials and 
associated wages with each step. Labor 
market information can be used to identify 
the specific occupational fields—and the 
jobs within these fields—that are in demand, 
provide family-sustaining wages and benefits, 
and offer opportunities for advancement 
within a region or state. Community 
colleges developing career pathways should 
collaborate with employers and industry 
partners to review labor market information 
and consult them to identify occupations 

for which training is needed." The questions 
in Figure 1.2 can guide the review of labor 
market information by bringing attention 

to in-demand skills and knowledge that 
employers are looking for, growth industries 
and occupations, as well as the training 
needed to access these. The specific questions 
are meant to serve as examples and can be 
modified to community colleges’ local needs 
and contexts. Community colleges seeking 
more information may wish to explore 
resources from Corporation for a Skilled 
Workforce, such as their toolkit that offers 
more in-depth guidance to understanding 
and applying labor market data.” 


Recommendation 1 


FIGURE 1.2. Sample uses of labor market 
information 


What skills are employers looking for? 


What are the skill requirements for different 
levels of positions, such as entry- or mid-level, in 
the industry? 


What training or education would an individual need 
for specific occupations or clusters of occupations? 


Which occupational areas are growing — now and in 
the future? 


Which industries are hiring — now and in the future? 


Are the jobs in these industries living-wage? 


What are the salary levels? 


Are there opportunities for job advancement? 


What factors can help/stop individuals from getting 
a job? 


What are working conditions like for specific 
industries? 


Source: Adapted from Career Pathways Toolkit: An 
Enhanced Guide and Workbook for System Development 
(2016) available at https://careerpathways.workforcegps. 
org/resources/2016/10/20/10/11/Enhanced_Career 
Pathways _Toolkit 


With a firm understanding of industry needs, 
community colleges are better positioned 

to align career pathways content and 
curriculum, structure, and credentials. This 
ensures students develop in-demand skills 
and knowledge, they have multiple successive 
entry and exit points, and they can pursue 
in-demand job opportunities at multiple levels. 
Beyond reviewing labor market information, 
a training gap analysis can help community 
colleges and their partners identify priorities 
for developing or improving training available 
through career pathways. The Training Gap 
Analysis Matrix featured in Figure 1.3 can 
guide both analysis and discussion of topics 
such as preferences community colleges 

and their partners have for training content, 
delivery models, and length. The information 
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Figure 1.3. Training gap analysis matrix 


Industry: 


Recommendation 1 


Career Pathway: 


Key Training; Preferred 

Key Occupations Content Training 
in the Pathway | (competency Delivery 
areas) WM exe (= (59) 


{lower-skill 
occupations} 


Preferred 
Training 


Associated 
Equipment 
or Software 
Require- 
ments 


Other Notes 
(instructor 
competencies, 
etc.) 


Required 
Credentials 


{middle-skill 
occupations} 


{higher-skill 
occupations} 


Source: A Practical Guide to Developing Career Pathways, httos://www.michigan.gov/documents/ted/ 


R5_5_MICPFramework_20180730_629836_7.pdf 


for the individual career pathways can be 
organized according to lower, middle, and 
higher skill occupations, facilitating an 
overview of the training gaps at different levels 
of the career pathway. 


It is essential that employers and industry 
partners provide input into the design 

of training and curriculum included in a 
pathway. An important role for employers 

is to keep community colleges informed of 
skill needs and what credentials prepare 
students for in-demand jobs. This may 
require dedicated time for continuous 
program alignment work. Through employer 


engagement strategies, such as advisory 
committees, community colleges can engage 
employers and industry partners in pathway 
development and periodic reassessment of 
pathway competencies and credentials.” 
Further, ongoing assessment of pathways 
provides opportunities to add additional 
in-demand training options." Finally, 
collaborating with industry partners provides 
an opportunity for mutual accountability 
between community colleges and employers 
in that colleges commit to developing skilled 
workers that employers commit to employing. 
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2. Define and create accessible pathways 


with clearly defined entry and exit points. 


Career pathways options that are clearly 
defined enable students to see how the 
education and occupational training along 
a pathway can lead to specific occupations 
or clusters of occupations. This is especially 
important since today’s students rarely 
follow a linear path to program or degree 
completion, and often stop-in and stop-out 
along the way. Pathways are most accessible 
to students when they have clearly defined 
entry and exit points. Contextualized 
basic skills instruction, offered through 
either bridge programs or integrated 
training models such as those described in 
Recommendation 2, can facilitate entry 
into career pathways. When pathways align 
with in-demand occupations within an 
industry sector, colleges can optimize the 
chance that students acquire the skills and 
knowledge needed to earn a credential and 
gain employment. 


Well-designed career pathways provide 
students with opportunities to continuously 
build or “stack” content knowledge and 
in-demand credentials as they progress 
along the pathway. Stackable credentials 
promote vertical movement to the next- 
higher-skill job in a sector, with each new 
credential providing a new set of skills to 
master. The Office of Career, Technical, 

and Adult Education’s Introduction to 
Stackable Credentials offers an overview of 
stackable credentials and how they relate to 
career pathways and programs of study 
(see Figure 1.4). Programs of study 

and career pathways share many of the 


Recommendation 1 


same attributes. The two terms are used 
interchangeably in many state and local 
applications. Both terms are defined 

in federal laws supporting workforce 
development and career preparation. 


Pima Community College’s Pathways to 
Healthcare Program created five pathway 
programs (medical office, nursing, medical 
and physician support, emergency medicine, 
and other).'* The steps of each pathway were 
designed to be stackable, so as to ensure 
students acquired the necessary prerequisites 
and associated credentials as they advanced 
along the chosen pathway. Each level of a 
pathway, as shown in the Highlights from 
the Field below, culminates in associated 
credentials that recognize student progress. 
For example, a student entering the first 

step in the nursing pathway must complete 
instruction and clinical work (and pass a 
licensing test) to earn the nursing assistant 
credential in Level 1, which prepares the 
student either for employment as a nursing 
assistant or to advance to the patient care 
technician program in Level 2. 


A student entering the pathway with a 

higher skill set than is needed for Level 1, 

as measured by the TABE or ACCUPLACER 
assessment, might enter the nursing pathway 
at Level 2. Conversely, a student with very 
low skills would start the pathway in a pre- 
pathway bridge program. The pre-pathway 
bridge offerings in Pima’s Pathways program, 
including a 10-week College Readiness Course 
or access to a self-paced College Readiness 
Lab, prepare students to be successful in 
Level 1 coursework. 


WWC 2021007 


Designing and Delivering Career Pathways at Community Colleges | Recommendation 1 | 12 


Recommendation 1 


Figure 1.4. Pathways to Career Readiness and Advancement 


Programs of Study and Career Pathways share many of the same attributes. The two terms are used 
interchangeably in many state and local applications. Both are defined in Federal Law. 
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combination of rigorous, | CAREER ADDITIONAL CREDENTIALS 
oa ADVANCEMENT ~ CREDENTIALS At these milestones the 
eo learner may advance to 
Solvers: the next-higher-skill job 
Attributes: A in the sector for which 
e Industry alignment they have trained, and/ 
* Secondary and CAREER BACHELOR’S or continue in or reenter 
postsecondary ADVANCEMENT _~ DEGREE the |saming patiway 
credential attainment to pursue additional 
credentials. 


Enables entry and 
advancement in 
specific occupations or 
occupational clusters 
Education in the same 
context as workforce 
preparation 
Acceleration of 
educational and career 


*These stackable 
credentials may: 

¢ Include preparation for 
industry certifications. 
Articulate to bachelor’s 
degree programs. 

Be obtainable by HS 
students through dual 


We 


advancement credit. 

¢ Preparation for success 
in secondary/ CAREER 
postsecondary EXPLORATION 


pel ak ti an Begins no later than 8th 
ashe a vie AN grade and is an integral 
Seen eel vees => HS DIPLOMA part of instruction for: 
ada cesl — SUSE Uae AI Career and Technical 
A PROGRAM OF STUDY Education (CTE) 
(POS) is a coordinated, 
nonduplicative sequence Integrated Education 
of academic and technical content at the and Training (IET) 
secondary and postsecondary level. Adult Basic Education 


Attributes: EXPLORATION cae 

: ult Secondary 
Industry alignment . Education (ASE) 
Postsecondary credential : 

: ee English as a Second 
Multiple entry/exit points Language (ESL) 
Academic, technical, employability skills 
Challenging standards 
Progressive content specificity 
Career guidance 


>: 
(2) 
> 
A 
m 
m 
A 


Source: U.S. Department of Education, Office of Career, Technical, and Adult Education (2021). 
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Recommendation 1 


Highlights from the Field YA 


Mapping health career pathways at Pima Community College 


Under the Health Profession Opportunity Grants (HPOG) demonstration program sponsored by the 
Administration for Children and Families, PIMA Community College (PCC) and the Pima County One-Stop 
(PCOS) mapped 16 existing healthcare occupational training programs into five career pathways, each 
incorporating a ladder that enables students to obtain stackable credentials. The five pathways were Medical 
Office, Nursing, Medical and Physician Support, Emergency Medicine, and Other pathways. 


The Pathways to Healthcare program was intentionally designed to be easier for advisors to explain and for 
students to understand how completing one credential could lead to a higher credential and higher paying job. 
Within each pathway, programs were designated as entry level (Level 1), mid-level (Level 2), or advanced level 
(Level 3). Programs within each pathway were structured so that Level 1 programs included all pre-requisites 
for Level 2 ones, and Level 2 programs included pre-requisites for Level 3 ones. For example, Nursing Assistant 
(Level 1) instruction and clinical work was completed before that for Patient Care Technician (PCT, Level 2), 
which enables a nursing assistant to also work in hospitals rather than just in nursing homes. 


With the Pathways to Healthcare program, PCC tied student learning outcomes to expected outcomes in 
the industry, with each level indicating greater mastery. Level 1 programs, for example, equipped students 
with basic occupation-specific knowledge—such as recall of data—and comprehension, such as understand 
the meaning, translation, and interpretation of instructions and problems. Level 2 programs required 
application of knowledge and skills, such as applying what was learned in the classroom to new workplace 
situations. Level 3 programs focused on (i) analysis—for example, breaking something down into its parts; 
(ii) synthesis—such as creating something new by putting together different ideas; and (iii) evaluation—for 
example, assessing the value of ideas or actions for a particular situation. 


(Gardiner et al., 2017) 
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Another approach to developing stackable 
credentials is creating short-term trainings 
within existing programs. This allows 
students to earn shorter-term credentials 

that facilitate entry-level employment while 
also allowing credits earned in the short- 
term trainings to apply toward subsequent 
credentials within a program. Wisconsin’s 
Making the Future TAACCCT 2 consortium 
grant, for example, led sixteen technical 
colleges, in partnership with employers and 
workforce development groups, to embed 
short-term certificates or credentials within 
longer-term “parent” programs, allowing 
students to receive targeted technical training 
in a short period of time.” Students who “job- 
out” of the pathway to seek employment in 
their field of study can continue to progress 


Recommendation 1 


toward a technical diploma or associate’s 
degree because they can apply their previous 
embedded credential courses and credits 
toward the parent program." 


Community colleges can also design career 
pathways that enable students to pursue a 
career lattice. Unlike a vertical career ladder, 
a lattice can involve lateral moves in the 
labor market. When presented in a graphic, 
lattices—like career pathways maps—can help 
students visualize career opportunities and 
understand the postsecondary education 
and training needed to ensure their skills 
align with a wide variety of labor market 
opportunities. As the example in Figure 1.5 
shows, students completing an associate’s 
degree in early childhood education (ECE) are 


Figure 1.5. Sample Pennsylvania Early Learning Keys to quality career lattice 
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Source: https://grisnetwork.org/sites/all/files/conference-session/resources/MaureenMurphySession215PP TasPDF.pdf 
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qualified for a range of positions in a variety 
of settings, each of which could lead to a 
more advanced career step with additional 
education and training and/or experience. 


Community colleges can support latticing 
by offering a core curriculum for a particular 
sector and then allowing students to take 
different directions along the pathway upon 
completion of the core requirements. For 
example, the Clean Energy Center program 
offered by the Kern Community College 
District (KCCD), one of the largest community 
college districts in the United States, 
required all students to take a six-week 
foundational training course that introduced 
them to the tools and equipment used 

in the field. This course was designed to 
provide students with a set of technical and 
transferrable skills that they could apply 
beyond the renewable energy sector."? In 
establishing the center, KCCD’s Workforce 
Development Division drew on its long 
history of partnering with local Workforce 
Investment Boards (WIBs) and employers 

to offer non-credit workforce training (see 
also Recommendation 5 on sustaining 
partnerships with industry partners). 


Community colleges can also offer students 
mobility to move in and out of the labor 
market at each level of credentialing. 
Returning to the nursing pathway at Pima 
Community College, for example, instead 

of moving immediately to the patient care 
technician program in Level 2, a student 
who earned a nursing assistant credential in 
Level 1 might work for a while first.2° That 
mobility also can allow students to consider 
shifting to another, related field of study 
when they do return to school. It is important 
to remember career pathways often are not 
linear, as the path chosen by an individual 
student is dependent upon their individual 
needs, including personal circumstances and 
professional goals. 


Recommendation 1 


When defining and creating a given career 
pathway, the expert panel recommends 
community college administrators and staff 
need to do a thorough assessment of course 
requirements, offerings, and scheduling to 
ensure students can access the courses they 
need in the timeframe they need them.” 
Institutional research may shed light on 
students’ course-taking patterns and inform 
pathway design. Students approaching 
transition points along a pathway may 
require extra supports to make sure they do 
not encounter barriers to progression (see 
Recommendation 4). 


Some community colleges have developed 
career pathways by starting with an assessment 
of existing programs. As one example, the 
Patient Care Pathway program developed by 
Madison Area Technical College, a community 
college with extensive experience operating 
bridge programs, adapted and linked existing 
programs at the college to create three 
1-semester academies offering low-skilled 
students an accelerated pathway into their 
chosen healthcare program.” The shared aim 
of the three re-designed academies was to 
accelerate student progression by combining 
basic skills and occupational courses (see 
Recommendation 2 on integrated basic 

skills training). 


Allow sufficient time for curriculum 
development and approval, but seek 
accelerated approval when feasible. 


If new courses will be required for a pathway, 
community colleges will need to allow 
sufficient time for curriculum development 
and approval. The procedures and timelines 
for developing a course vary by state, by 
institution, and whether the course is 
intended to be for credit or not. However, 
given advancing technologies and the need to 
respond more rapidly to employer demands, 
institutions are employing a variety of 
approaches to accelerate curriculum design 
and approval. 
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Some institutions have found success 
incubating pathways and courses through 
their continuing education or non-credit 
programs, and later seeking approval to 
offer these pathways as part of new or 
existing degree programs. Colleges that use 
this approach should remember to consider 
non-credit to credit articulation processes 
that will allow students who went through 
non-credit workforce training programs to 
move into degree programs without having to 
retake courses.” 


The panel encourages college administrators 
to carefully review their program 
development steps and processes to 

identify institutional roadblocks that could 
potentially be removed. They can also 
consider whether there are any fast-track 
program approval options in their state, 
which may facilitate their efforts to respond 
more quickly to industry changes. Some 
states have delegated curriculum approval 
processes to the local level. In California, for 
example, the Curriculum Committee of the 
Academic Senate of California Community 
Colleges developed guidance for local senates 
to ensure effective curriculum approval 
processes (see Figure 1.6). 


Regardless of how long curriculum approval 
can take, it is important to allocate faculty 
and staff sufficient time to engage in 

quality curriculum development. One 

way to do this is to release faculty from 
teaching responsibilities to collaborate with 
employers. It is also important to ensure 
faculty and deans have time to coordinate 
with each other to ensure that all the pieces 
of a career pathway fit together and that one 
step along the pathway adequately prepares 
students for subsequent steps. 


Further, the panel recommends assessing the 
scope of curriculum development required 
and selecting a reasonable amount of new 
curriculum to develop at one time.” When 
enhancing an existing program, as opposed to 


Recommendation 1 


developing an entirely new program, ongoing 

engagement with employers may allow faculty 
to make small, incremental changes over time 

that would not require external approval. 


FIGURE 1.6. Sample recommendations for 
ensuring effective curriculum approval processes 


Review and evaluate the effectiveness of local 
curriculum processes. 


Ensure that local senate purview over curriculum 
and the connection between the local senate and 
the curriculum committee are well-understood. 


Ensure that the curriculum committee structure 
includes a diverse array of faculty, academic 
administrators, students, and staff that can provide 
a variety of expertise and perspectives without 
weakening faculty primacy over curriculum. 


Ensure that the process for the initiation of new 
curriculum and revisions to existing curriculum is 
clear, the technical review process is streamlined 
and effective, and curriculum committee meetings 
are run efficiently. 


Streamline the curriculum approval process 

by ensuring a sufficient frequency of curriculum 
approval opportunities by the curriculum committee 
and the governing board, establishing an expedited 
approval process for time-sensitive proposals, 

and providing individual colleges in multi-college 
districts autonomy over their curriculum. 


Provide professional development at the 
appropriate level for faculty, administrators, 
students, and staff, with more detailed training 
provided to those most closely involved with the 
local curriculum process. 


Advocate for sufficient resources to support 

the work of the college curriculum team, including 
reassigned time or additional compensation, and 
for the provision of ongoing funding and access to 
professional development opportunities. 


Ensure that faculty initiators of distance education 
proposals are provided with professional 
development on effective practices for ensuring 
regular and effective contact and compliance with 
accessibility requirements. 


Source: Academic Senate of California Community 


Colleges, https://www.asccc.org/papers/ensuring-effective- 
curriculum-approval-processes-guide-local-senates 


WWC 2021007 


Designing and Delivering Career Pathways at Community Colleges | Recommendation 1 | 17 


In making any changes, it is important to 
ensure they are aligned with the most current 
information about what students need to 
succeed and advance in jobs. 


The DACUM (“Developing a Curriculum”) 
process is one structured approach to 
developing a new curriculum-or revising an 
existing one—with the input of employers and 
industry leaders. DACUM incorporates the use 
of a focus group in a facilitated storyboarding 
process to capture the major duties and related 
tasks included in an occupation as well as the 
necessary knowledge, skills, and traits. Once an 
occupational profile is created and validated, 

a task analysis is conducted to further define a 
job. Both are used to develop a curriculum.” 
Developing a “Business and Industry 
Leadership Team” (BILT) is one way to extend 
beyond the DACUM process to continually 
involve a group of employers by engaging them 
to annually review—and suggest changes to— 
existing courses within a program.” 


Connect students to pathways that align 
with their knowledge, skills, educational 
interests, and career goals. 


Students interested in pursuing a career 
pathway can benefit from recruitment 
and intake procedures that are designed 


Figure 1.7. The Project QUEST intake process 
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to assess their knowledge, skills, and 
educational and employment background 
as well as their career aptitude and goals.”” 
These opportunities help students better 
understand their strengths, areas of growth, 
and aspirations, thereby helping them 
determine—as they better understand 
pathway options and relevant career 
opportunities along the way—whether those 
pathways are a good fit for them. 


Intake processes may also be used to assess 
student interest and readiness to commit to 
attending courses, both of which enhance 

a student’s engagement and increase the 
likelihood that they will be successful in 
progressing along a pathway. Intake processes 
may include applications, skills assessments, 
career interest assessments, one-on-one 
counseling, and career coaching.”* For 
example, Project QUEST in San Antonio, 
implemented by a partnership between 

two local community organizations and 

the Alamo Community College District, 
involves a rigorous six-stage intake process 
(see Figure 1.7), within which each student 
meets one-on-one with an occupation-specific 
counselor to evaluate the student’s career 
aptitude and interests, personal background 
and aspirations, family and work situation, 
education and employment history, and 


Source: Roder & Elliott (2018) 
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placement test scores, among other topics. 
Students develop an academic plan as part 
of the process and leave with a roadmap for 
completing a certificate or degree program in 
the healthcare industry. 


Potential Obstacles and the 
Panel’s Advice 


OBSTACLE: Industry needs can change rapidly, 
making it challenging to maintain alignment 
between pathways curriculum and employer needs. 


PANEL’S ADVICE: Industry needs often 
change at a faster pace than community colleges 
are equipped to respond. This challenge is 
particularly acute in emerging industries where 
competencies, labor market demands, and 
external funding sources to support training for 
specific populations are in flux.” 


To be responsive to changing industry needs, 
such as advances in technology, the expert 
panel recommends college administrators and 
staff should identify opportunities to remove 
barriers that prevent faculty from adapting 
their programs in a timely fashion, such as long 
program or curriculum approval processes or 
limited opportunities for deep engagement with 
employers and industry partners. 


Establishing and maintaining strong, regular, and 
meaningful connections with employers is one of 
the most effective ways for community colleges 
to ensure their pathway programs remain 
relevant and responsive to industry needs.*° 
Employers can be involved in frequent reviews 
of curriculum. They can also be asked to identify 
skills that are transferable to multiple career 
paths in the same industry, thereby affording 
students some flexibility in choosing occupations 
as they advance along a career pathway. 


Active employer partner teams, such as 
advisory committees, can provide regular 
feedback channels between colleges and 
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industries to ensure all stakeholders are informed 
of changes or emerging needs within local or 
regional educational and employment contexts. 
Quarterly meetings that allow for the discussion 
of changes in industry needs and trends make it 
easier for colleges to incrementally modify small 
portions of their curriculum as needed. 

See Recommendation 5 for more information 
on how community colleges can develop and 
leverage partnerships with employers and 
industry leaders. 


OBSTACLE: Students encounter barriers to entry 
and progression along career pathways. 


PANEL’S ADVICE: Staff and faculty can ensure 
that students are able to enter and progress 
through pathways by identifying, monitoring, 
and addressing barriers that students encounter. 
Upon pathway entry, some students lack the 
math, reading, and/or writing skills required to 
successfully complete college-level work. Barriers 
related to career pathways entry also include 
narrow application windows, long waitlists, 
prerequisite requirements, and assessment 
burdens, and institutions lacking practices that 
are culturally responsive or responsive to the 
needs of adult learners with busy schedules and 
multiple responsibilities to juggle. 


The panel recommends community colleges 
should develop program structures that ensure 
smooth transitions along career pathways. One 
way to do this is to design bridge programs, 
typically preparatory courses in basic skills, that 
provide core instruction to many related career 
pathways programs (latticing), which increases 
the number of options students have when they 
complete bridge programming. Instructors of such 
courses can also encourage students to progress 
along a career pathway by incorporating short 
introductory segments of occupational training 
in their curriculum, or by inviting CTE faculty to 
deliver introductory lectures to bridge students.*! 
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Other deliberate strategies for helping students 
advance into career pathways upon completing 
basic skills courses include intensive advising*? and 
financial incentives.** Students may also benefit 
from supports to help them prepare applications 
or study for pre-program assessments. 


The expert panel believes community college 
staff should pay particular attention to 
transitions where students may experience 
challenges progressing to the next step ona 
pathway or when re-entering the pathway. 
These transition points may offer practical 
opportunities for building in new supports, such 
as additional advising or navigation services 

to connect students to relevant academic, 
non-academic, career, and financial supports 
(see Recommendation 4). Paying attention 

to the particular needs of diverse student 
populations, including language, income and 
other personal and familial circumstances, is 
important to enabling these students to make 
the transitions successfully. Active monitoring 
of student progress often may require enhanced 
institutional research capabilities, and 
community colleges are highly encouraged to 
make these commitments. At the same time, 
community colleges should approach their career 
pathways programming with an eye towards 
equity, so as to counter some of the barriers 
faced by underrepresented students.** 


OBSTACLE: Initial steps on some career pathways 
lead to low-wage jobs. 


PANEL’S ADVICE: Balancing the short- and 
long-term needs of both employers and students 
can be challenging for community colleges 
designing and implementing career pathways. 
Consistently involving employers and industry 
professionals in career pathways development is 
one way to routinely identify family-sustaining 
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employment opportunities in industries relevant 
to the college’s postsecondary occupational 
offerings and align the college’s program 
offerings accordingly. 


The panel believes it is important for community 
college administrators and staff to have candid 
conversations with employers about wages 
prior to career pathways development. Will 

the resulting wages equal the effort put in by 
students pursuing the pathway? Will they be 
sufficient for students to repay college debt, if 
applicable? Will they sustain a living-wage job 
that leads to future promotions and increasingly 
higher pay along a career pathway? Community 
colleges should avoid contributing to the 
development and/or delivery of certain types of 
training that does not yield living-wage jobs, and 
also restrain from flooding the market with an 
oversupply of candidates for low-wage jobs. 


When an initial step on a career pathway 

trains students for a low-wage job—such as 
Certified Nurse Assistant on a healthcare 
pathway-— the expert panel suggests it is 
important for community colleges and employers 
to consider whether that initial job will prepare 
completers for a next step on the pathway, and 
whether that next step will lead to employment 
in a higher-skilled, better-paying job. It is also 
critical for college faculty and staff to make 

sure students are aware of the likely salary at 
each step along a career pathway. Including 
information about salaries when mapping 

career pathways is one way to remind colleges, 
employers, and students of the full range of 
opportunities along the pathway. As shown in 
Figure 1.8, it can be useful to map out the career 
pathway and the corresponding employment 
opportunities and wages associated with different 
levels of the pathway. 
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Figure 1.8. Sample career pathway map 


CAREER OPPORTUNITIES 
Architectural Engineering Design 


Prepares students for employment in the field of residential design or a related technical field such as drawing for product 
manufacturers, engineering or design firms. 


Prior graduates have entered engineering technician positions in computer aided drafting and design (CAD), project 
management assisting, residential design and site planning and developing, truss design, and performing structural 
calculations and computations for engineering of wood beams and joints. Students participate in realistic training 
activities as a part of their educational experience. 


STARTING OPTIONS FOR MORE 


STARTING ae i eMail Gk Seer nisere / Some College INFORMATION 
¢ Dual credit courses ° 
eo RE SSeS * GED, ESL, Adult Basic Skills (ABE), -BEST ==» Contact the 


Apply to Clover Park * 1st Career / 2nd Career Architectural 
Technical College * Seeking Professional Credentials Engineering 
* Continuing Ed Department 
ENTRY-LEVEL CERTIFICATE OPTION EMPLOYMENT AND WAGES 
¢ Architectural CAD Drafting ¢ Auto CAD Technician 
- |-Best Program for ESL/ABE (100% applicable to degree) aa Drafter 


¢ 3-quarter program 
¢ Dual Credit Articulation Agreements 
¢ May transition from Trades Academy 


u 


ARCHITECTURAL ENGINEERING DESIGN ASSOCIATE DEGREE EMPLOYMENT AND WAGES 

° AAT & AAS-T ¢ Civil Engineering Technician 
* 6-quarter program ap Architectural & Civil Drafter 
¢ Internships * Cost Estimator 
¢ Dual Credit Articulation Agreements ¢ Entrepreneurship Opportunities 
¢ Day and Hybrid Classes 

BACHELOR'S DEGREE EMPLOYMENT AND WAGES 

¢ Embry Riddle University - Technical Management Degree * Architect 

¢ The Evergreen State College = Upside Down Degree =>. Entrepreneurship Opportunities 


¢ Lake Washington Institute of Technology - Applied Design 
¢ Check with the receiving 4-year institution for transfer of credit 


Source: http://washingtoncareerpathway.org/c/published/1275/cptc-architectural-engineering.html 
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Recommendation 2: Deliver contextualized or integrated basic 


skills instruction to accelerate students’ entry into and successful 
completion of career pathways. 


Students entering career pathways have diverse Students may benefit from contextualized 
educational and employment backgrounds as instruction, which includes “instructional 

well as knowledge and skill levels, which may strategies designed to more seamlessly link the 
drive their learning preferences and needs. learning of foundational skills and academic 
The evidence related to this recommendation or occupational content by focusing teaching 
suggests that administrators, faculty, and staff and learning squarely on concrete applications 
should strive to be responsive to the different in a specific context that is of interest to the 
instructional needs of the students to support student.”* Figure 2.1 offers an example of how 
their successful completion of career pathways. to design contextualized reading instruction by 
Integrated basic skills instruction allows students engaging with career and technical instructors, 
to start from their current skills levels and as well as employers and industry partners, in 
develop the foundation in math, reading, and developing industry aligned learning goals and 
writing (“basic skills”) they need to enter and lessons plans. 


pursue occupational coursework. For an example 
of integrated basic skills instruction, see the 
Highlights from the Field on page 24. 


Figure 2.1. Example of contextualized reading instruction 


Building on the skills identified by the CTE faculty, Ms. Franeta, the ESL Instructor, developed some learning 
objectives for the basic skills instruction as part of the Programa de Carpinteria Fina, a fine woodworking 
program at Laney College. For example, in the contextualized reading class, some of the learning goals 
developed include: 


» Apply previewing methods to reading technical texts. 

» Practice inferring and critical thinking skills in reading and writing technical material. 
» Recognize and use correctly vocabulary of the trades. 

» Analyze and practice English word building. 

» Describe and report work situations or conditions in technical language. 


During the program development stage, Ms Franeta, sought authentic materials to use in contextualizing her 
English course and the other academic courses. The career and technical instructors helped her to identify 
several resources: articles in Fine Woodworking magazine, textbooks, and even specialized websites such as the 
Architectural Woodwork Institute (http://awinet.org/). These instructors also explained what they do in class and 
offered advice on how the basic skills instructors could support the technical classes. 


Several employers and industry specialists also volunteered advice about the priority skills that it would be helpful 
for students to learn in addition to the woodworking skills. They provided input on job expectations and related 
skills (e.g., calculating skills and strong customer service). 


Working with the CTE faculty, Ms. Franeta identified eight units around some common skills themes, such as 
organizing details in the reading. She then worked collaboratively with the CTE faculty to develop lesson plans for 
these units that incorporated the career-focused content. Once each of the lesson plan units was complete, she 
also developed a syllabus for, in this case, the reading course. A similar process was followed for the 

other courses. 


Source: https://tcall.tamu.edu/docs/ContextualizatingAdultEdInstructionCareerPathways.pdf 
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Students may be more likely to engage with basic 
skills instructional materials that draw upon 
occupational subject matter that interests them. 
Thus, contextualization can make basic skills 
training more relevant to students, improve 
student motivation, and promote transfer and 
application of newly acquired skills. When most 
effective, contextualized or integrated basic skills 
instruction provides accelerated on-ramps to 
career pathways by closing skill gaps and allowing 
students to see how basic skills will apply in 
future occupational training and employment. 


Contextualization can take two forms:*° 


¢ Contextualized basic skills instruction 
broadly refers to instructional strategies that 
make explicit connections between learning 
of basic skills and occupational content. 
Examples of contextualized instruction 
include using examples from nursing 
(medicine dosage) as part of teaching basic 
math skills or using excerpts of textbooks from 
occupational areas when teaching reading 
comprehension. Contextualization is primarily 
woven into the assignments and examples 
used to teach basic skills. 


¢ Integrated basic skills instruction refers to 
the strategy where basic skills instruction and 
occupational content instruction are delivered 
at the same time, often by a team of instructors, 
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one basic skills and one postsecondary 
occupational. Incorporating basic skills into 
postsecondary occupational instruction helps 
students to accelerate their progress along a 
career pathway because they can begin taking 
occupational coursework at entry. 


Delivering contextualized or integrated basic 
skills instruction can involve considerable 

effort on the part of instructors. Development 

of contextualized curriculum and course 
materials, as well as alignment of teaching 

roles and instruction styles, requires time 

and resources. Successful contextualization 

of basic skills requires the commitment of 
administrators to provide opportunities and 
resources for instructors to develop and 

deliver a contextualized or integrated basic 

skills curriculum. Community colleges can 
consider partnering with local community-based 
organizations and offer adult education programs 
to help deliver the contextualized or integrated 
basic skills instruction. 


WWC staff and the expert panel assigned a 
moderate level of evidence to Recommendation 
2, based on nine studies of interventions that 
include contextualized or integrated basic skills 
instruction. Seven of the studies*” meet WWC 
group design standards without reservations, and 
the other two studies** meet WWC group design 
standards with reservations. 


Table 4. Results of Meta-analysis for Recommendation 2 


Statistically 


Yefatiicer-laiar-lale| 


Statistically 
YfeTalniter-talar- ae | 


Outcome Domain 


positive 


Credit accumulation @ 


Indeterminate ral=XeF-14hY-) 


Postsecondary degree 
attainment 


Industry-recognized credential 


attainment e 


Note: Across the nine studies, there were findings in three outcome domains even though all five educational outcome 
domains (see Box 1) were relevant for this recommendation. None of the nine studies included findings that meet WWC 
standards in the following outcome domains: (1) academic performance; or (2) technical skill proficiency. 
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In nine of the 10 studies used to support and delivering contextualized or integrated 
Recommendation 2, contextualized or integrated basic skills instruction. All figures and 

basic skills instruction is a major or secondary mentions of specific community colleges in 
component of the tested intervention. See Recommendation 2 are offered as examples only 
Appendix C for a detailed rationale for the level and should not be read as endorsements 

of evidence assignment for Recommendation 2. of specific products or approaches. 


This section describes strategies, examples, 
and tools that can support community college 
administrators, faculty, and staff in designing 


Highlights from the Field J 


Integrated Basic Education and Skills Training 


Developed by the Washington State Board for Community and Technical Colleges (SBCTC), Washington 
State’s Integrated Basic Education and Skills Training (I-BEST) program is a nationally known program that 
aims to increase access to and completion of college-level occupational training in a variety of in-demand 
occupational areas. 


Delivered within a structured career pathway, the SBCTC designed I-BEST to provide the first step on an 
articulated career pathway in a range of occupations, particularly to students who need additional basic 
skills instruction to access and succeed in occupational training. 


The signature feature of I-BEST is team teaching by a basic skills instructor and an occupational instructor 
during at least 50 percent of occupational training class time. SBCTC required team teaching, but allowed 
flexibility in how instructors approached their teaching, including how they defined their roles within the 
classroom, planned their time together, and established the rapport to work collaboratively. 


In addition to team teaching of basic skills and occupational training, I-BEST programs can also provide 
separate non-credit basic skills courses designed to complement the occupational courses. In these support 
courses, the basic skills instructors review occupational material with the students, help them with their 
homework, and help them prepare for tests and quizzes. 


(Glosser et al., 2018) 
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How to Carry Out the 
Recommendation 


The guidance below is informed by the 
studies that support the recommendation as 
well as the expert panel’s knowledge of 
strategies and resources available to help 
implement the recommendation. 


1. Assess CTE instructional materials to 
identify the math, literacy, and writing 
skills students need to successfully 
progress along career pathways, 
assess students to determine their skill 
levels, and identify opportunities to 
contextualize instruction. 


Reviewing course textbooks, assignments, 
and other materials from courses offered 
at various points along a career pathway 
is one way to determine the skills students 
need in order to be successful and master 
the occupational content in a course. 
When conducting this review, the expert 
panel suggests instructors should review 
course materials to ensure the content 

is appropriately aligned to the basic skill 
levels of incoming students and identify 
opportunities to teach or reinforce basic 
skills using examples and concepts that 
are relevant to occupations along the 
pathway. There are several resources freely 
available to help instructors plan and 
implement contextual teaching and 
learning approaches.*? 


Community college faculty and staff can 
better design instructional supports for 
students if they know the levels of skills 
required to understand the course content 
as well as the skills levels of their incoming 
students. Data from intake assessments of 


basic skills proficiency can inform what types 


of skills instruction students need, as well as 


how much instruction they need to help them 
better understand course content.*° Students 
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re-entering pathways after being out of school 
for an extended period of time may also 
benefit from additional supports related to 
study skills and time management.” 


As students progress through career 
pathways instruction, ongoing assessments— 
both formal and informal—may help faculty 
and staff identify opportunities to provide 
additional instructional supports. For 
example, in response to emergent student 
needs, one career pathways program added 
a supplemental instructor to lead review 
sessions and offered group and one-on-one 
training.” Instructor-led study groups and 
College Readiness classes, which included 
modules on basic reading, writing, and 
math skills and covered college and career 
success topics, such as time management, 
test taking, and understanding how to 

learn, are examples of another community 
college’s multi-pronged response to emergent 
student needs.* 


Determine strategies for teaching 
basic skills that are appropriate for 
the student population served and the 
resources available. 


Two common strategies are (1) pre-pathway 
bridge programs that offer contextualized 
basic skills curricula, for example, teaching 
math by calculating dosages in ways that are 
relevant to careers along the pathway“ and 
(2) integrated instruction, where students 
begin an occupational training course 

on a pathway, but are team taught by an 
occupational instructor and a basic skills 
instructor.* Bridge programs are offered 

just before or as the first step of a career 
pathway and can be designed to help students 
master basic skills needed to support their 
transition into pathway courses, especially 
when contextualization helps them see direct 
connections and applications of those skills in 
the occupations they are pursuing. Integrated 


WWC 2021007 


Designing and Delivering Career Pathways at Community Colleges | Recommendation 2 | 25 


instruction, on the other hand, embeds basic 
skills instruction into the technical content 
and is applied at each pathway step. Both 
strategies can provide students the basic skills 
needed to succeed along career pathways, but 
their design and implementation vary greatly. 


When selecting the best strategy for providing 
basic skills instruction, community college 
administrators will need to consider the 

time and resources that would be required 

of staff and instructors implementing the 
selected strategy. Bridge programs can be 
offered on campus or at off-site locations 

run by community-based organizations, but 
initial and ongoing planning or coordination 
between basic skills instructors and 
occupational instructors is required to ensure 
the basic skills instruction is relevant to the 
occupational content that will be covered in 
subsequent courses. 


Curriculum specialists can help identify topics 
that can be contextualized in a basic skills 
class and support instructors to help them 
learn the content. They can also emphasize 
the importance of contextualized instruction 
and provide supports to instructors to ensure 
uniformity and consistency across courses 
within a pathway.*° 


If implementing a team teaching model 
for courses with integrated instruction, 
select a model that best fits your 
institution and faculty. 


Community colleges should consider 
various team teaching models to ensure 
the one they select is most appropriate 
and feasible for the college and its faculty. 
Options for team teaching may range from 
monitoring, where one teacher is responsible 
for instruction while the other circulates 
around the classroom monitoring students’ 
needs, to shared teaching duties, where 
the occupational instructor and basic skills 
instructor each focus on their own area of 
expertise.*” Considerations for selecting a 
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specific model include instructional needs 
of students, abilities of instructors, and 
how adaptable the subject area is for 
team teaching. 


The evaluation of Washington State’s 
Integrated Basic Education and Skills Training 
(I-BEST) program identified three different 
approaches to team teaching that community 
colleges could consider adapting to their 
local context: (1) basic skills instructor 
participates as an active student; (2) basic 
skills instructor delivers part of the content 
during a designated part of each occupational 
class; and (3) basic skills and occupational 
instructors jointly deliver class content (“co- 
teaching”).** These three approaches are 
described in Figure 2.2. 


Provide professional development and 
collaborative planning time to support 
team teaching. 


Devoting time and resources to professional 
development for both occupational 
instructors and basic skills instructors may 
help ensure they are adequately prepared 
for their team teaching assignments.*® The 
evaluation of I-BEST, implemented at over 
30 community and technical colleges across 
Washington State, reported on the learning 
curve several faculty experienced when they 
were newly assigned to team teach, noting 
“for many occupational instructors, I-BEST 
courses were their first experience teaching 
low-skilled students, whereas basic skills 
instructors needed to learn occupational 
content quickly.”°° 


Adequate, ongoing planning time for faculty 
to coordinate their teaching styles and the 
instructional content can promote more 
successful team teaching. The expert panel 
believes occupational instructors can learn 
pedagogical approaches from basic skills 
instructors; conversely, basic skills instructors 
can ensure basic skills are applied in more 
meaningful ways when they understand the 
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Figure 2.2. Sample team teaching models 


Observation of Basic Skills Instructor Participating as 
an Active Student 


The second course in a two-quarter introductory machining 
sequence was a math-intensive course that emphasized “shop 
essentials.” The objective of the course was to familiarize students 
with measurement tools that they would be using in the shop. The 
class was primarily lecture based and led by the occupational 
instructor. The instructor explained the use of different tools and 
parts, the application of geometry concepts to the machining tools, 
and demonstrated measurement techniques using the tools. 


The basic skills instructor frequently summarized and 

reiterated key points and clarified concepts that seemed to 
confuse students. The basic skills instructor regularly posed 
questions to the occupational instructor for further clarification, 
such as differences in certain cutting techniques and when certain 
techniques should be used. This resulted in regular dialogue 
between instructors. 


Both instructors responded to student questions, answering 
depending on their expertise. When answering questions, the 
basic skills instructor focused primarily on math concepts and 
terminology, whereas the occupational instructor described tools 
and measurement strategies with some reference to 

math concepts. 


Observation of Basic Skills Instructor and Occupational 
Instructor Co-teaching 


The Sustainable Office Skills class was structured around the 
creation of a portfolio for each individual student. It included both 
independent and small group work, as well as some interactive 
content often based on assigned readings. 


For the group work, instructors spent a substantial amount of 
time orienting students to the task and explaining the goal of the 
project. Students worked in pairs or groups of three to develop a 
sample question to ask an employer to improve sustainability in 
the workplace, such as “Have you considered using a recycling 
bin?”. After working in groups, the instructors ask students to 
share their ideas, and the basic skills instructor defined terms 
that English language learners may have found difficult to 
understand. Students worked independently to write individual 
30-second elevator speeches about themselves in preparation for 
job interviews. The occupational skills and basic skills instructors 
also role-played as job candidate and potential employer, 
demonstrating interview and job readiness skills. 


Instructors were highly integrated in delivering content, 

with limited differentiation of the basic skills and occupational 
instructor roles. The two instructors interacted constantly 
throughout the class, with both clarifying concepts and delivering 
instructions on activities and assignment expectations. The basic 
skills instructor asked clarifying questions both to students and to 
the other instructor, provided feedback to student responses, and 
explained classroom tasks. 


Source: Glosser et al. (2018) 
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Observation of Basic Skills 
Instructor Delivering Content During 
an Occupational Class 


Office Procedures was a required course in 
the Clerical Assistant sequence. It focused 
on business English, time and resource 
management, telecommunications 
knowledge and phone etiquette, 
organizational skills, basic financial tasks, 
administrative skills, oral presentations, 
business terminology, and conflict 
resolution in an office setting. 


Roughly half the observed class was an 
interactive large group discussion, with 

the remainder evenly split among lecture, 
independent work, and small group 

work. The lecture component began with 
the basic skills instructor giving a brief 
presentation on test-taking strategies. This 
was followed by a seminar-style discussion 
led by the occupational instructor. Students 
had the option to work independently or 
with a partner on a worksheet activity 
about time management, followed by group 
work in which small groups of students 
were tasked with developing strategies for 
effective time management given scenarios 
that were assigned by the instructor. The 
two instructors were purposeful in drawing 
out students’ personal experiences and 
attempting to make them relevant to the 
course content. 


The occupational instructor delivered 

most of the content, and the basic skills 
instructor offered additional clarification or 
examples. The two instructors traded off 
in assisting students or calling on students 
to speak during activities. The content was 
not overly technical, so the occupational 
and basic skills instructors both spoke 
about general success strategies, skills, 
and concepts. 
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contexts in which those skills need to be 
applied. Further, instructors across several 
sites of the Accelerating Connections to 
Employment (ACE) program, designed to 
improve employment outcomes for low-skilled 
workers through formal partnerships between 
WIBs and community colleges, reported 
planning and implementing a co-teaching 
model allowed them to identify students’ 
problems faster, such as learning disabilities 
or difficulty understanding certain concepts, 
and then work one-on-one to address them.*! 
ACE was implemented at nine sites across 
Maryland, Texas, Connecticut, and Georgia. 


5. Assess whether basic skills instruction 
(contextualized or integrated) is delivered 
in a manner that is resulting in students 
achieving their learning goals. 


Program directors can use a number 

of approaches to assess the delivery of 
contextualized basic skills instruction in 
bridge programs or integrated courses offered 
on campus. For example, they can observe 
classroom instruction, conduct interviews 
with faculty or program staff, or collect 
feedback from students through surveys or 
focus groups.” 


Administrators can use administrative data to 
assess whether the delivery of contextualized 
basic skills instruction is increasing the 
progression rate of students in a pathway. 
For example, administrators and staff can 
determine how many students successfully 
transitioned from a pre-pathway bridge 
program to pathway courses. Administrators 
and staff can periodically consult instructors 
to make sure students are acquiring the 

skills necessary to be successful in their 
occupational courses and able to progress 
along the pathway. If not, they may 

consider incorporating additional basic 

skills preparation into courses, and might 
also add additional academic supports for 
students still demonstrating skills gaps** (see 
Recommendation 4). 


Recommendation 2 


At transition points along a career pathway, 
summative assessments can be used to 

make sure students have mastered the 

basic skills necessary to be successful in 

the next occupation or training program 
along the pathway. Where applicable, 

these assessments should align with college 
admission tests to ensure students can earn 
scores that help them avoid being placed into 
developmental coursework. 


Potential Obstacles and the 
Panel’s Advice 


OBSTACLE: Developing materials for 
contextualized basic skills instruction can be 
time intensive. 


PANELS ADVICE: Instructors in both 
individual teaching and team teaching scenarios 
may face challenges developing materials for 
contextualized basic skills instruction. Instructors 
who are part-time or are otherwise overbooked 
may have insufficient time to adapt course 
content to support contextualized instruction, 
assignments, and assessments. Or the team may 
have misaligned schedules, making it difficult 
for them to engage in joint planning. The 

panel believes administrators should consider 
providing release time or other incentives 

to faculty to allow them to jointly develop 
instructional materials or coordinate team 
teaching opportunities. 


Instructors may be able to save time by 

seeking out existing resources that can be 
adapted to their local courses and context. 
There are several online repositories of open 
educational resources, such as SkillsCommons 
(see Figure 2.3), that faculty and staff can use 
to identify relevant resources and information 
for course development. The materials are 
tailored to specific industries and occupations. 
If resources permit, curriculum specialists can 
provide tips for reviewing and selecting such 
materials. Finally, local employers and industry 
partners may be willing to share non-proprietary 
materials that can double as course materials. 
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OBSTACLE: Team teaching can be more costly. team teaching. Braided funding approaches 
may also help community colleges support team 


PANEL’S ADVICE: Team teaching can be 
resource intensive, and it does not always fit 
neatly into faculty workloads or contracts. 

The expert panel suggests community colleges 
committed to increasing retention and 
progression rates for students participating in 
career pathways should look across their networks 
for examples of other colleges that have found the 
funds and workload flexibility required to support 


Figure 2.3. SkillsCommons 


author of the resources. 


‘SKILLS COMM@NS 


The U.S. Department of Labor’s Trade Adjustment Assistance Community College and Career Training 
(TAACCCT) program created a free and open online library called SkillsCommons containing free and open learning 


materials and program support materials for job-driven workforce development. The $1.9 billion investment by the 
U.S. Department of Labor produced a comprehensive collection of workforce-related Open Educational Resources 
(OER) created by over 700 community colleges across the nation. The OER can be found, reused, revised, retained, 
redistributed, and remixed by an individual, institution, and industry for FREE with the proper attribution to the original 


teaching.** Community colleges unable to afford 
team teaching should consider how to support 
contextualization in a single course taught by 
one instructor by providing that instructor 

with appropriate professional development or 
with course planning/coordination between 
occupational and basic skills instructors. 


Users can search for 
information or materials 

by industry, credentials, or 
material type. The ‘Advanced 
Search’ feature allows users 
to conduct more targeted 
searches if they have more 


Showcases Connect Contribute 


View our showeases of course material Connect with Workforce Development Upload OER and workforce development 
‘makeovers, open courseware and project ‘communites, technology partners and career -—_matertials and visit our: 
coutcomes. services partners. when you need help. 


Explore Workforce Development 
Resouces 


(Open Educational 


Users can explore SkillsCommons by industry by clicking 
on the wheel graphics. When the user clicks on a section of 
the wheel, they get an expanded presentation of a specific 
industry. Pop-up boxes appear when users mouse over 
sections of the wheel. Clicking on a name in the pop-up box 
leads the user to a list of available materials for that specific 
industry. An accessible version lists all of the industries with 
hyperlinks to materials within each industry. 


Source: http://www.skillscommons.org/ 


specific search criteria in mind. 


The Support Center includes detailed 
information on how to discover, contribute 
and manage, or revise and reuse materials in 
SkillsCommons. It also includes information 
on licensing and accessibility requirements, 
as well as frequently asked questions. Users 
can browse support topics through “quick 
links” or use the search function. 
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Recommendation 3: Offer flexible instructional delivery schedules 


and models to improve credit accumulation and completion of non- 
degree credentials along career pathways. 


Students must balance their education and 


training with jobs, families, and other obligations. 


Often, that can make traditional course formats 
and schedules problematic. Flexible delivery 
of instruction, through non-traditional course 
times and/or online or self-paced courses, 

can help students combine college with other 
commitments to access, and progress along, 
career pathways. 


Flexible course scheduling includes offering 
evening or weekend courses or multiple sections 
of a course across a range of days, times, 

and locations. Block scheduling can be used 

to coordinate or group courses in ways that 
consolidate the amount of time students need to 
spend on campus. 


Technology can be used to vary, blend, or 
accelerate course formats to accommodate 
students who prefer online or hybrid learning 
formats or who might benefit from self-paced 


learning modules.* 


Awarding credit for prior learning, offering 
co-requisite courses, and/or offering competency- 
based courses, programs, and assessments can 
accelerate or compress students’ time to earning 
credentials. When carefully designed, these 
approaches can also help to ensure that students 
acquire—and demonstrate mastery of—both 
technical and basic skills as they progress along a 
career pathway. 


WWC staff and the expert panel assigned a 
minimal level of evidence to Recommendation 3, 
based on six studies of interventions that include 
flexible instructional delivery models. Five of 

the studies** meet WWC group design standards 
without reservations, and the other study*’ meets 
WWC group design standards with reservations. 
An additional three studies® include flexible 
instructional delivery models but do not report 
any educational outcomes. 


Table 5. Results of Meta-analysis for Recommendation 3 


Statistically Statistically 
Yefaliicer-laiar-lale| Yio Taluicer-talar- Tae} 

Outcome Domain positive Indeterminate rat=XeF-1ahY-) 
Credit accumulation e 
Postsecondary degree 

: ® 
attainment 
Industry-recognized credential e 
attainment 
Academic performance @ 
Technical skill proficiency @ 
Note: Across the six studies, there were findings in all five educational outcome domains (see Box 1). 
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In all seven of the studies used to support 
Recommendation 3, flexible instructional 
delivery models are a secondary component 
of the tested intervention. See Appendix C for 
a detailed rationale for the level of evidence 
assignment for Recommendation 3. 


This section describes strategies, examples, 

and tools that can support community college 
administrators, faculty, and staff in offering flexible 
instructional delivery and course acceleration. 

All figures and mentions of specific community 
colleges in Recommendation 3 are offered 

as examples only and should not be read as 
endorsements of specific products or approaches. 


How to Carry Out the 
Recommendation 


The guidance below is informed by the studies 
that support the recommendation as well as 
the expert panel’s knowledge of strategies 

and resources available to help implement 

the recommendation. 


1. Assess the institutional readiness 
and resources needed to offer flexible 
scheduling and/or instructional 
delivery models. 


Setting up flexible scheduling and course 
offerings in multiple formats requires 
coordination and logistics. For example, 
academic calendars and schedules may need 
to be modified to accommodate flexible 
course scheduling. Designing innovative 
instructional delivery models requires similar 
levels of coordination as well as resources 
for instructional technology and professional 
development for the instructors who will use 
the technology. 


Institutional capacity assessments can help 
community colleges think about which things 
they need to work on first when designing 
more flexible course schedules and models. 
Faculty willingness and readiness to teach 
courses on nights and weekends, as well 
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as resources and capacity to develop and 
deliver materials for alternative course 
formats, should be part of the assessment. 
Instructional designers and faculty may need 
to review information technology platforms 
to determine the extent to which they can 
support online or hybrid courses. 


Tailor flexible course schedules and 
instructional models to diverse student 
needs and instructional preferences. 


Students have different capabilities, needs, 
barriers, and learning preferences that 
influence their ability to engage, to be 
retained, and to be successful along a career 
pathway. Community colleges can use various 
strategies to maximize flexible scheduling, 
such as block scheduling, evening and 
weekend course offerings, and self-paced 
online modules.*® These should be carefully 
aligned with, and respond to, the diverse 
needs of students. 


Block scheduling reduces time required on 
campus, whereas evening courses can 
accommodate students who are employed.“ 
The New Visions Self-Sufficiency and Lifelong 
Learning Project found it necessary to offer 
basic skills instruction in three-hour time 
blocks that repeated three times daily to 
meet students’ varying work, childcare, and 
transportation needs.” The program, which 
targets TANF recipients, was provided by 

a partnership between Riverside County 
Department of Public Social Services and 
Riverside Community College, which serves 
over 30,000 students in Riverside, California. 


Self-paced online modules can help students 
address basic skills gaps while starting or 
preparing to start occupational courses.” 
Flipped classroom models use technology to 
help students learn new content outside of 
class and then use class time to reinforce the 
concepts and explore them in more depth 
with their instructors and peers. 
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Provide training and support to 
instructors developing flexible 
instructional delivery models. 


Developing hybrid, online, and flipped 
classroom formats requires skill sets that 
extend beyond those used in developing 
more traditional courses. Faculty designing 
and delivering flexible instructional models 
for career pathways must learn to leverage 
instructional technology while remaining 
attentive to student engagement strategies.© 
They must also ensure course learning 
objectives, instruction, and assessment align 
to industry needs, as identified during the 
creation and implementation of the career 
pathways (see Recommendation 1). 


Successful design and delivery of alternate 
course formats may require additional 
funding for instructional technology, as 

well as time and resources for instructors to 
collaborate with instructional designers to 
ensure their new hybrid or online courses are 
effective. To ensure they will be successful, 
the expert panel believes administrators 
need to allocate sufficient resources to course 
scheduling and design efforts. Perkins V 
provides funds colleges can access for new 
professional development programs for 
instructional staff. 


Faculty may seek advice from peers through 
their professional networks or online 
repositories of open educational resources, 
such as SkillsCcommons or MERLOT. These 
repositories include development tools, 
online course materials and modules, syllabi, 
and materials for hybrid courses, to name 

a few. As resources permit, instructional 
designers can be paired with faculty 
developing and delivering alternative course 
formats.® Faculty may consult EDUCAUSE’s 
“7 things you should know about” series, 
which offers a collection of concise briefs 
that summarize emerging technologies and 
practices and answer seven questions to help 
instructors understand whether and how to 


Recommendation 3 


adopt them. The briefs on Open Education 
Content, Collaborative Learning Spaces, 

Video Communication, and Assessing Online 
Team-Based Learning might interest faculty 
exploring flexible instructional delivery 
models and communication and collaboration 
tools to adopt in their courses.” The 
EDUCAUSE library also includes a collection 
of resources organized by topics such as 
Online Course Development Planning and 
Online Teaching Strategies.®* Faculty will need 
the support of administrators and institutional 
support staff in order to be successful in 
designing and delivering flexible instruction. 


Use acceleration strategies, such as 
prior learning assessments and 
competency-based education, to 
reduce the time between students’ 
pathway entry and their attainment of 
non-degree credentials. 


Acceleration strategies can be applied to 
structural as well as instructional aspects 
of career pathways. These acceleration 
strategies include offering contextualized 
or integrated basic skills training (see 
Recommendation 2), awarding credit for 
prior learning, or engaging students in a 
competency-based education (CBE) model. 
Students entering pathways with skills gaps 
may benefit from a variety of approaches 
to accelerating any necessary basic skills 
instruction. As just one example, Madison 
Area Technical College, a college with 
many years of experience designing and 
implementing bridge programs, created 
sectoral bridge programs to package and 
compress basic skills and occupational 
training to prepare students for a specific 
occupational field. The goal was to 
accelerate the students’ entry into the 
college’s healthcare diploma and degree 
programs by shortening basic skills training 
to one semester.®? Colleges can also partner 
with other organizations to offer brief 
preparatory courses to help students test 
out of remedial courses.” 
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Prior learning assessments (PLAS), one 
mechanism for awarding credit for prior 


learning, help students move from the 
non-credit- to the credit-bearing parts of a 
career pathway.” For example, the Northeast 
Resiliency Consortium, a collaboration of 
community colleges, industry groups, and 
workforce agencies, leveraged a consortium- 
wide focus on PLA to develop regional PLA 
standards that included both internal and 
external PLA options (see Figure 3.1), and that 
led several colleges to modify and formally 
approve their institutional PLA policies. 


Figure 3.1. Strategy for developing regional prior 
learning assessment (PLA) standards 


The Northeast Resiliency Consortium (NRC) 
developed regional prior learning assessment (PLA) 
standards that its colleges assessed against their 
existing policies and processes. Regional PLA 
standards were developed through an iterative 
process among the NRC consortium leads, college 
faculty and staff, and national experts. These PLA 
standards are intended to expand opportunities for 
students to receive credit for prior learning. 


At several colleges, these regional PLA standards 
went under review by college leaders and 
administrators, and most colleges are confident 
that the standards will align with their existing 
standards or will be approved as new standards 
for the college. Some colleges expressed concern 
about institutional barriers, for example faculty 
resistance or limited institutional resources, to 
moving forward with the regional PLA standards. 
In general, however, colleges are adapting and 
aligning the regional PLA standards within their 
unique institutional contexts. 


As part of the implementation of PLA standards, 
colleges worked with nationally recognized experts 
and received on-campus training for faculty and 
staff. These sessions were often also attended 

by faculty and staff not directly working on their 
college’s NRC grant. 


Source: Price et al. (2017) 
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Another acceleration strategy is Competency- 
Based Education (CBE). CBE is a curriculum 
design in which students acquire and 
demonstrate their knowledge and skills by 
engaging in learning exercises and activities 
that target clearly defined competencies. 
Credentialing is based on mastery of targeted 
competencies, rather than on seat-time, 
clock-hours, and face-to-face instruction. In 
this way, CBE allows students the opportunity 
to progress through their coursework in an 
accelerated manner. 


Salt Lake Community College, a large 
two-year college serving over 60,000 
diverse students across ten campuses, 
applied TAACCCT grant resources toward 
incorporating both CBE and PLAs to make 
its curriculum “more open to the larger 
environment and successful in transitioning 
students into employment and further 
education.”” A key purpose of converting 
curriculum to a CBE model was to reduce 
time-to-degree, reduce cost of attendance, 
and emphasize self-paced and self-directed 
learning that would motivate students 

to complete credentials. Administrators 
developed CBE support materials for faculty, 
including training, a handbook, and new 
policies and procedures. Community colleges 
interested in offering CBE may wish to 
consult the Competency-Based Education 
Network, especially its Quality Framework for 
CBE Programs and the accompanying Quality 
Framework User’s Guide.” 
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Recommendation 3 


Highlights from the Field YA 


Competency-Based Education 


Through a TAACCCT grant, Salt Lake Community College’s School of Applied Technology and Technical 
Studies converted 20 high priority programs of study (POS) to competency-based education (CBE) with the 
goal of credentialing students upon their mastery of competencies and completion of POS, and of placing 
less emphasis on seat-time, clock-hours, and face-to-face instruction. 


The college’s CBE model used a hybrid delivery approach, incorporating self-paced, online learning 
modules, and hands-on labs and competency-based assessment. Another important aspect of the project 
was to enhance Recognition for Prior Learning (PLA), allowing students to progress through their POS in an 
accelerated manner. 


The process of converting POSs to CBE required collaboration between faculty, administration, and 
instructional designers in the college’s Online & eLearning Services Division. Faculty could consult with the 
Instructional Design unit that employed professionals trained in instructional technology, CBE-instructional 
design, and assessment to convert applicable portions of their curriculum into the CBE format. The 
conversion also involved ongoing consultation through Program Advisory Committees, drew upon the 
experience of an expert in PLA, and provided professional development for faculty and staff. 


(Bragg et al., 2018) 


Continuously monitor and respond to 
students’ needs regarding flexible course 
offerings and delivery modes. 


Administrators, instructors, and staff 

in career pathways should continually 

assess whether students are able to access 

the courses they need to make progress 
toward credit accumulation and credential 
attainment. This includes ensuring required 
courses are offered frequently, and at 

varied times. Community colleges must 

also consider whether students have the 
necessary hardware and internet connectivity 
to connect to and participate in courses 

that require remote lectures, labs, or other 
activities. The expert panel that authored the 
Using Technology to Support Postsecondary 
Student Learning Practice Guide 

suggested “instructors, department heads, 
and administrators share responsibility in 
monitoring and evaluating the effectiveness 
of the course formats and delivery modes 


used to engage and teach learners.”” 

Many Learning Management Systems now 
incorporate surveys that continuously 
monitor students’ progress and challenges, 
including scheduling and student attendance 
problems. These types of student data may 
indicate unmet needs that could 

be addressed. 


As needed, community colleges may use 
data on student progress and outcomes to 
inform adjustments to when, how frequently, 
and in what format courses are offered.” 
Academic program reviews, which involve a 
comprehensive analysis of program quality 
and are required for accreditation, provide 
an opportunity to examine a wide variety of 
data about a program. These data include 
historical enrollment patterns, external 
demand, program cost analysis, program 
outcomes, and feedback on areas for 
continuous improvement. 
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Potential Obstacles and the 
Panel’s Advice 


OBSTACLE: Flexible instructional delivery 
schedules and models demand high levels of 
institutional flexibility and ongoing coordination 
among instructors and administrators. 


PANEL’S ADVICE: The design and 
implementation of effective instructional 
delivery schedules and models that are flexible 
require commitment and time of instructors. 
However, the expert panel cautions the work 
of the instructors will not be enough if they 

do not also have support from administrators, 
support staff, and institutional infrastructure 
in place. Specifically, the panel notes the need 
for administrators to include resources in their 
budgets to explicitly support the design and 
implementation of flexible delivery models 
and schedules, especially through professional 
development and an investment in technologies 
to support flexible delivery models. 


Alternate instructional delivery approaches, 
such as hybrid, online, or mobile courses, will 
likely require instructors to change the way 
they design and teach their courses. When 
administrators emphasize the importance of— 
and commit to supporting—flexible instructional 
delivery schedules and models, they can set the 
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expectation that instructors and staff must work 
together to design and deliver instruction that 
best meets the diverse needs of their students. 
Data on student needs and outcomes might 
help in making the case to faculty for alternative 
course schedules and formats, but should also 
be used to understand patterns in student 
performance and to identify students who need 
additional supports—whether instructional or 
technological—to succeed in classes that are not 
taught in-person. 


OBSTACLE: Some courses may not lend 
themselves well to online formats. 


PANEL’S ADVICE: Though courses in some 
content areas may require a lot of hands-on 
training, such as welding, manufacturing, and 
nursing, some aspects of these courses may 
still be compatible and even enhanced by an 
online learning environment or the use of high- 
fidelity physical models. For example, health 
care training may use sophisticated life-like 
mannequins in realistic patient environments 
to create structured scenarios requiring specific 
and essential actions that are predictable and 
reproducible for all students in a course, which 
can reduce the amount of time students must 
spend in clinical experiences, thereby offering 
more students the opportunity to engage in 
these experiences.” 
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Recommendation 4: Provide coordinated comprehensive student 


supports to improve credit accumulation and completion of 
non-degree credentials along career pathways. 


Students often need to navigate a variety of are unavoidable as students progress along their 
academic and non-academic challenges that career pathway-—from intake to completion. 

can affect their ability to complete coursework 
and progress toward earning a credential. These 
challenges include choosing the right program 

of study and career, balancing education with 
family and work obligations, and covering 
tuition costs and related educational expenses. 
Providing comprehensive student supports in a 
coordinated fashion helps students to be resilient 


to these challenges. When program faculty and staff are attentive to 
students’ emergent needs, it may allow them to 
be more responsive in offering tailored supports 
that will keep students on the path to successfully 
completing their studies. 


To help ensure that student supports are readily 
available, delivered just in time, and easy to 
access, consider assigning faculty and staff with 
clearly defined roles and lines of communication. 
The panel believes that greater level of 
coordination could help avoid gaps and overlaps 
in providing these supports. 


The evidence suggests and the expert panel 
believes colleges should provide a broad range 
of academic and non-academic, career, and 
financial support services. These may include 


proactive academic advising, mentoring, Providing comprehensive student supports can 
coaching, counseling, career navigation, and be costly. Both internal and external partnerships 
financial aid, as well as referral to other support can be leveraged to provide and enhance student 
services. The panel recognizes most community support services. Braided funding, using multiple 
colleges currently offer students many of the funding sources to cover the costs of student 
supports featured in the studies that informed support services, is one strategy for covering the 
this Practice Guide. The panel believes these costs of the supports provided.” 

student supports should be delivered both 

one-on-one and in small group settings, and WWC staff and the expert panel assigned a 

that the supports should be more intentionally moderate level of evidence to Recommendation 
integrated into the student experience so they 4, based on 15 studies of interventions that 


include comprehensive student supports. 


Table 6. Results of Meta-analysis for Recommendation 4 


Statistically Statistically 


Yrefaniicer-laiar-lale| YfeTalniter-Talar- ale} 
Outcome Domain positive Indeterminate ral=XeF-1ahY-) 


Credit accumulation e 


Postsecondary degree 
attainment 


Industry-recognized credential 


attainment e 


Note: Across the 15 studies, there were findings in three outcome domains even though all five educational outcome domains 
were (see Box 1) relevant for this recommendation. None of the nine studies included findings that meet WWC standards in 
the following outcome domains: (1) academic performance; or (2) technical skill proficiency. 
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Twelve of the studies® meet WWC group 

design standards without reservations, and the 
other three studies” meet WWC group design 
standards with reservations. An additional 
study® includes comprehensive student supports 
but does not report any educational outcomes. 


In 14 of the 15 studies used to support 
Recommendation 4, comprehensive student 
supports are a major component of the tested 
intervention. See Appendix C for a detailed 
rationale for the level of evidence assignment for 
Recommendation 4. 


This section describes strategies, examples, 

and tools that can support community college 
administrators, faculty, and staff in offering 
comprehensive student supports. All figures 
and mentions of specific community colleges 

in Recommendation 4 are offered as examples 
only and should not be read as endorsements of 


specific products or approaches. 


Comprehensive Student Supports 


Highlights from the Field 


Recommendation 4 


How to Carry Out the 
Recommendation 


The guidance below is informed by the studies 
that support the recommendation as well as 
the expert panel’s knowledge of strategies 

and resources available to help implement 

the recommendation. 


1. Conduct an inventory of available 
supports and clearly define which 
college departments are responsible for 
delivering specific supports. 


Student support services may be provided by 
a large network of professionals, including 
those within a community college and 

those in the surrounding community. For 
this reason, the expert panel suggests it 

is important to conduct an organizational 


Project QUEST, developed and implemented by Communities Organized for Public Service and Metro 
Alliance, provides comprehensive support and resources to help students complete occupational training 
programs at local community colleges and professional training institutes, pass certification exams, and 


obtain jobs in targeted industries. 


The support services include: 


financial assistance to cover tuition and fees for classes, books, transportation, uniforms, licensing exams, 


and tutoring; 


remedial instruction in math and reading to help individuals pass college placement tests; 


counseling to address personal and academic concerns and provide motivation and emotional support; 
referrals to outside agencies for assistance with utility bills, childcare, food, and other services, as well as 
direct financial assistance with other supports on an as-needed basis; 

weekly meetings that focus on life skills, including time management, study skills, critical thinking, and 
conflict resolution; and 

job placement assistance, including help with writing resumes and interviewing, as well as referrals to 
employers that are hiring. 


Project QUEST career counselors work with participants from the time they are accepted into the program 
until they obtain a job. Students are required to attend weekly meetings and submit class attendance sheets 
signed by instructors in order to continue receiving support. Another important aspect of the QUEST model 
is that it targets well-paying occupations in strong sectors of the local economy and employs staff dedicated 
to developing relationships with employers and to providing participants with assistance in obtaining a job. 


(Roder & Elliott, 2018) 
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review of the services offered and clearly ¢ Career and employment supports 
define who is responsible for them. including employment workshops, 
career navigation, resume workshops, 
mock interviews, alumni network 
events, and assistance with job or 
internship placements. 


A resource mapping exercise can be an 
important first step toward identifying the 
student supports that are currently in place 
as well as service gaps, as they relate to 
projected student needs. ¢ Financial supports including tuition 
support, support for other education- 
related expenses (e.g., books, fees for 
certification exams, work boots, or 
uniforms), funds for transportation or 
childcare, as well as guidance on how 

to identify and access external funding 
opportunities (e.g., Pell grants and other 
types of financial aid). 


The resource map may cover the following 
types of student supports: 


¢ Academic supports such as academic 
advising, tutoring, textbook lending, 
additional study sessions, and instructional 
support in math/writing, among others. 


¢- Non-academic (personal) supports 
such as assistance with life challenges that 
are interfering with academic progress, 
including family or work obligations, food 
security, housing issues, transportation, 
and childcare. 


Figure 4.1 highlights some of the questions 
that can guide the development of the 
resource map-—from an initial assessment 

of the student support services currently 
provided towards a subsequent assessment of 


Figure 4.1. Resource mapping questions 


i 
a 


Source: Alliance for Quality Career Pathways (2014). Shared vision, strong systems: The Alliance for Quality Career 
Pathways Framework Version 1.0. Washington, DC: Center for Law and Social Policy. 
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the resources available to remedy any gaps in Gathering information as part of resource 
or unmet need for additional student services. mapping provides an opportunity to engage 
with different partners that provide student 
services—both within and outside of the 
community college—and to clarify roles 

and responsibilities for service delivery. 
Community colleges can use the resource 
mapping template featured in Figure 4.2 to 
create an overview of the different support 
services provided and to determine who is 
responsible for delivering which supports.” 


The supports provided by community 
colleges and their partners need to align 
with and respond directly to the most 
pressing needs of students.*! A central part 
of resource mapping involves determining 
the timing, frequency, and intensity of the 
supports provided, as well as relating these 
to the projected student needs. This allows 
for an early assessment of the potential costs 
associated with the supports provided. 


Figure 4.2. Career pathways resource mapping template 


This resource mapping template is provided to assist partners in identifying available resources to support the 
development and operation of education, training, and related programming for career pathway, as well as 
potential resource gaps. 


IXY EL CsXe | 
Resources: 
Organization Key Primary ST-¥ aYd (042) Targeted cS) oF: [ot RS) LL Contact 
Name Programs ST-V ad [ox=4) Restrictions ede) LUI E-La ey atcy Training Information 
Equipment, 
etc. 


Post-Secondary Education Partners 


Workforce Development Partners (WIOA partners, training providers, etc.) 


Community-Based/Human Services Partners 


Industry Partners 


Other Partners 


Source: A Practical Guide to Developing Career Pathways, https://www.michigan.gov/documents/ted/R5_5 
MICPFramework_ 20180730 629836 _7.pdf 


WWC 2021007 Designing and Delivering Career Pathways at Community Colleges | Recommendation 4 | 39 


Finally, it is important to consider how 
students will access the student supports. 
Resource mapping provides an organizational 
overview of who will offer academic supports, 
non-academic (personal) supports, career 
and employment supports, and financial 
supports to the students. Further, colleges 
should consider how to engage internal and 
external partners so that they can collectively 
identify gaps in support services and improve 
how students access available services. 


After identifying available supports and 
providers, the expert panel notes it is 
important for community colleges to develop 
processes for referring or connecting 
students to the different supports, especially 
those beyond the community college. A range 
of community-based organizations may be 
available to help students overcome barriers 
to successfully completing a career pathway, 
and staff at community colleges will be 
better equipped to facilitate referrals to these 
organizations when they are aware of the 
services they offer. 


Assess students’ needs and connect one- 
on-one to provide them with relevant 
supports, from intake to program 
completion and beyond. 


Accessing student supports can be difficult, 
especially when a broad range of student 
support services are made available 

by a network of partners within and 
outside the community college. For some 
students, navigating these services can be 
overwhelming. Many community colleges 
have started to embrace the importance 

of wraparound supports for students.* 
Institutions committed to offering such 
supports may need to encourage and help 
faculty to work more closely with student 
support staff. While FERPA regulations may 
prevent the exchange of information about 
individual students, the expert panel believes 


Recommendation 4 


community college administrators should 
consider whether there are workshops or 
trainings they can offer faculty to make them 
more aware of some of the challenges faced 
by the student populations, as well as the 
resources available to help students address 
those challenges. Pathway intake processes 
and assessments that are holistic can be 
used to match students to the supports 
they need to be successful as they start and 
progress along a career pathway. Today’s 
postsecondary learners are, taken together, 
more diverse—and representative of the 
nation as a whole-than ever before. College 
staff, including counselors and advisors, 
may vary in their capacity to recognize and 
respond to the challenges faced by an ever- 
changing student body. Training, including 
implicit bias training, can help ensure 
educators are better-positioned to guide 
students in seeking the most relevant and 
helpful supports. 


The expert panel recommends that during 
the career pathways recruitment and intake 
process, student support staff should conduct 
a comprehensive needs assessment with 
each student. This needs assessment should 
be holistic, focusing on both academic 

and non-academic (personal) issues.** This 
may include the student’s educational and 
employment background, family and work 
obligations, learning preferences, financial 
situation, and any other aspect of the 
students’ experience that may influence 
their progression along the career pathway. 
The primary aim of the assessment is to 
ensure students are matched with and 
connected to the right supports to help them 
be successful. The assessment can be done 
during a well-designed onboarding process, 
connecting each student with a designated 
advisor or career navigator. The Community 
College Research Center offers resources 

in Redesigning Community College Student 
Onboarding Through Guided Pathways that are 
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designed to help community colleges think 
about onboarding in more holistic ways to 
ensure students explore career and program 
options, choose a program direction, and 
develop a full-program plan.* 


Students should leave the intake process with 
a clear overview of the steps and available 
supports along their chosen career pathway. 
In the Pathways to Healthcare Program at 
Pima Community College, a two-year college 
serving a largely nontraditional student 
population in Tucson, Arizona, advisors offer 
incoming students guidance on occupational 
program requirements and provide a 
program-specific checklist (“passport”) of 
steps students must complete, from pre- 
pathway enrollment through employment. 
Figure 4.3 shows Pima’s Participant Plan 
Checklist, which is introduced by advisors to 
students when they begin their program. The 
checklist includes key events and milestones 
that students should attend to as they 
progress through their career pathways. 


The plan was designed to serve as a blueprint 
that connects all program-related steps for 
students, and the checklist is a tool that 
advisors and other program staff can use to 
track students’ progress. 


3. Integrate available supports into the 
student experience. 


The expert panel believes faculty and staff 
should proactively provide student supports 
and modify supports appropriately as 
students progress through a career pathway. 
There are several ways of integrating and 
making student support services accessible 
along the career pathway. Embedding student 
support staff into pathway programs to 
continuously help students navigate training 
choices, provide support during training, and 
help them identify barriers to employment is 
one way to do this. 
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Figure 4.3. Sample participant plan checklist 


0 HPOG (program) information session 
C1 Intake eligibility/commitment appointment 


0 Pima County One Stop Workforce Development 
Specialist appointment 


CZ Referred to College Readiness (Y/N) 
C] TABE scores 
J ACT Compass scores 


OJ Mandatory Employability Skills Workshop 


01 Pima Community College advising (Student 
Services Advanced Specialist) 


OJ Center for Training and Development information 
session 


(_] Received program passport (Y/N) 

OJ Pima Community College credit: completed 
degree plan (Y/N) 
CJ Applied for FAFSA (Y/N) 


C1 Student Support Specialist meeting 
CO Passport completion 
1 Training Plan completion 
1 Training Plan approved 
1 Training completed 
1] Workshops: 
LJ 123 Work: Employment Transition Workshop 
(J Job Offer Academy 
] Resume Writing 
[] Interview Technique 
{-] Computer Workshop 
(_] Pesco Sage, a career assessment that tests 
academic skills, aptitudes, learning styles, 
work attitudes, temperament and work ethic 
1 Employment specialist meeting 
CO] Employed 


Source: Gardiner et al. (2017) 


Student support staff can also be embedded 
into individual college classes for a certain 
number of hours per week or term. For 
example, career counselors can on occasion 
attend classes and introduce job search 
strategies and resources as part of in- 

class activities.*° At some colleges, faculty 
introduce career navigators as part of their 
instructional teams, have them participate 
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actively in class activities, and integrate 
student support services into class exercises. 
By integrating student supports in this way, it 
is easier for students to see the important role 
of the supports and to understand how these 
supports can facilitate their progress and 
completion of the career pathway. 


Making student support staff accessible 

in the evenings and on weekends will 
enhance accessibility for students balancing 
work and family obligations. For some 
student populations, using technology and 
online formats can increase accessibility 
and nudge students to make better use 

of existing student supports. Virtual 
advising technologies can help colleges 
reduce costs while meeting increased 
demands for support. Instructors often use 
various Learning Management Systems, 
such as Blackboard, Canvas, D2L, and 
communication tools, such Microsoft 
Teams among others, to communicate with 
students about assignments, online tests, and 
other course-related matters. These online 
platforms can also be used to alert students 
about job fairs, tutoring services, and other 
course-related student supports. 


Monitor student progress, academic 
and non-academic needs, and supports 
accessed. 


Identifying needs and connecting students 
with relevant supports do not end at intake. 
The expert panel encourages all staff and 
faculty who engage with students throughout 
their program of study to be attentive to 
changes in students’ academic and non- 
academic barriers. They should watch for 
signs that students are struggling, such as 
failing grades, not turning in homework or 
assignments, and decreased participation 
in class. Upon detecting barriers, faculty 
and staff should refer or connect students 
to services that can help meet students’ 
emerging needs. This requires that faculty 
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be keenly aware of and understand all the 
support services available, and that faculty 
and student support staff work in tandem. 


Staff and faculty can ensure that students 
are able to progress along pathways by 
monitoring their progress and identifying 
barriers that students encounter. 
Coordination amongst staff and faculty 
supporting the same students is especially 
critical at transition points.*” The expert panel 
encourages staff to pay particular attention 
to progress at transitions between education 
and employment, where students may 
experience challenges re-entering the career 
pathway, as these phases may offer practical 
opportunities for building in new supports. 


There are several strategies for monitoring 
student progress. Some pathway programs 
have proactive advisors, who initiate contact 
with students several times during their 
progression along the career pathway; other 
programs also dedicate resources to provide 
follow-up supports to students after program 
completion.** Ongoing audits of a student’s 
progress can be facilitated by institutional 
data on course completion and progress. Data 
on attendance and progress from Learning 
Management Systems support this type of 
monitoring. Project QUEST and the Valley 
Initiative for Development and Advancement 
(VIDA) embedded student audits within 
mandatory weekly workshops facilitated by a 
designated counselor. Students were required 
to hand in class attendance sheets as part of 
these workshops; the sheets documented 
their progress in the coursework.*® 


There is a growing range of early alert 
systems, such as Starfish, that can be used 
to monitor student progress and to help 
student support staff identify students who 
are falling off track. Technological solutions 
can also be used to track how and when 
students access support services. As just 
one example, some community colleges are 
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using SalesForce, a customer relationship 
management tool, to track when and how 
students make use of support services. 


5. Consider hiring sector-specific career 
navigators. 


The expert panel encourages community 
colleges to be proactive about helping 
students identify the best career fit for 
them. Discipline- or sector-specific career 
navigators, advisors, mentors, or coaches 
can tailor their advising and career guidance 
to the occupational sector in which a 
student plans to seek employment.*! When 
implemented well, navigators and career 
coaches can positively affect a student’s 
retention in a program and completion of a 
credential. Navigators can provide students 
with individual coaching, career planning, 
and assistance navigating the transition 

to further education and employment.*” 
Navigators can also serve as an important 
resource in connecting students to necessary 
services.** Hiring additional staff to serve 
as navigators can be costly. Therefore, 
community colleges could consider 
redeploying existing advisors with a new 
focus on a particular industry sector. 


At one of the community colleges in the 
Northeast Resiliency Consortium, the career 
coach engaged students throughout the 
duration of the program to deliver proactive, 
personalized one-on-one advising. As 
students neared program completion, the 
career coach passed them on to one of two 
employer relations coordinators, who would 
then connect students with job opportunities. 
The career coach was an integral part of the 
support services team, and the program’s 
employer relations specialists reportedly 
relied on the career coach to identify students 
nearing completion to engage with job 
searching services.** 
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Potential Obstacles and the 
Panel’s Advice 


OBSTACLE: Advisers and other staff charged 
with delivering these supports have caseloads that 
are too large for them to be able to handle the 
comprehensive needs of students. 


PANELS ADVICE: Providing comprehensive 
student supports, especially when tailored to 

the individual needs of each student, can be 
costly in support staff time and resources. 
Advising structures and budgets often cannot 
support frequent one-on-one advising with large 
numbers of students. To the extent possible, 
facilitating group sessions for students can enable 
support staff to convey general information and 
address common needs and issues. Delivering 
this general information to a large group of 
students allows advisors more time for individual 
sessions. Creating a common place where 
concerns are raised in a protected environment 
also fosters peer relationships between students 
so that they can support one another inside and 
outside the classroom. 


As part of the Northeast Resiliency 
Consortium, one of the community colleges 
facilitated weekly group workshops on 
self-care and related resiliency-related topics, 
providing students a shared space to discuss 
personal and professional challenges and 
solutions related to their participation in the 
program.* To facilitate attendance, the sessions 
were offered immediately before required 
coursework. Group advising sessions were also 
offered by VIDA to support students learning 
from one another’s experiences and to foster a 
peer support network. 
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OBSTACLE: Some staff may lack knowledge and 
skills needed to design and deliver coordinated 
comprehensive student supports. Faculty may lack 
awareness of the value and existence of the array of 
student supports that are available. 


PANEL’S ADVICE: The expert panel believes 
community colleges should invest resources 

to develop faculty and staff competencies to 
coordinate and collaborate on the delivery 

of student support services. Professional 
development and/or professional credentialing 
of student support staff (especially in relation to 
counseling) should be considered. 


Colleges may consider investing in professional 
development to assist more faculty and staff in 
understanding the different student support 
services provided and how these services 
intersect with student enrollment, retention, 
and completion.” Actively engaging faculty and 
staff in the resource mapping exercise facilitates 
a stronger understanding of the types of services 
provided and when and how students may 
need to access these. Ideally, resource mapping 
exercises involve relevant faculty and staff. 


Another strategy is to develop and organize 
“student success teams,” an advising/ 
instructional team that includes a navigator or 
student success coach. Such teams allow for a 
more coordinated, and potentially integrated, 
approach to delivery of student support 
services.°** There are a variety of sources of 
information for implementing potentially 
effective student success teams. They include 
networks of CTE professionals, experts in the 
design of career pathways more generally, 
and peer colleges serving the career needs of 
similarly situated students. 


OBSTACLE: It is costly to deliver comprehensive 
supports to students throughout all phases of 
their learning. 


PANEL’S ADVICE: Community colleges can 
partner with community-based organizations 
and other local community partners, such as 
American Job Centers, to coordinate and cover 
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the costs of providing the needed student 
support services by combining different funding 
streams (braided funding). 


VIDA is a community-based organization in 
southern Texas that supports participants’ full- 
time enrollment in a college through intensive 
advising and financial support. It receives 
funding from local Economic Development 
Corporations and cities and counties in the 
region to cover the costs of training for VIDA 
participants.°? In a similar vein, Project QUEST 
combined funding from local government and 
the private sector to cover the costs of providing 
comprehensive student supports.!° 


OBSTACLE: Very few students choose to 
participate or receive services without proactive 
efforts of community colleges. 


PANEL’S ADVICE: Students may be unaware 
of student support services, unsure of how to 
access or make use of student supports, or even 
reluctant to seek out relevant supports. The 
expert panel recommends embedding utilization 
of student support services into the overall 
program experience. There are several ways 

to do this. One strategy is to integrate student 
support services into the existing coursework 
and student schedules. As part of the Northeast 
Resiliency Consortium, staff delivered support 
services—in the form of workshops on interview 
preparation, time management, study skills, 

and self-care—during required pre-program 
orientation sessions or in class during scheduled 
lecture times. These in-class sessions reached 
more students than out-of-class sessions did.'*! 


Another strategy to maximize utilization is to 
require students to utilize support services. In 
Project QUEST and VIDA, active participation in 
weekly group advising sessions was mandatory 
for all students in each program.'” The group 
sessions—facilitated by designated counselors— 
covered program expectations and participant 
responsibilities, budgeting and saving, time 
management, balancing work and family 
obligations, notetaking and study strategies, and 
job interview preparation. 
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Students participating in Project QUEST are 
required to participate in weekly Vision, 
Initiative, and Perseverance (VIP) sessions. These 
group sessions, facilitated by career counselors, 
focus on life skills, time management, study 
and test-taking skills, critical thinking, conflict 
resolution, and workforce readiness, among 
other topics. Students are also required to bring 
in signed course attendance sheets, allowing the 
counselors to monitor the students’ progress in 
occupational coursework.!™ 
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Community college faculty and staff should 
also identify opportunities to connect students 
to support services in off-campus settings, 
such as activities or events held by community- 
based organizations or through other social 
service agencies. 
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Recommendation 5: Develop and continuously leverage partnerships 


to prepare students and advance their labor market success. 


Career pathways that leverage employment- ¢ increasing the potential for job placement and 
focused partnerships offer students both advancement. 


classroom- and work-based experiences to 
acquire the skills needed to be ready for 

work on day one. The benefits of investing in, 
building, and deepening employment-focused 
partnerships include: 


Ultimately, improving student labor market 
outcomes benefits students, employers, 

and colleges alike. There is an incentive for 
administrators, faculty, and staff to work 
together with employment-focused partners to 


¢ improving the relevance and alignment of the design, implement, and continuously evaluate 
curriculum to employer or industry needs; career pathways. 

* expanding the opportunities for students to WWC staff and the expert panel assigned a 
engage meaningfully with employer partners moderate level of evidence to Recommendation 
through employer presentations, onsite visits, 5, based on 11 studies of interventions that 
work-based learning opportunities, and career include employment-focused partnerships. 
fairs; and Seven of the studies'** meet WWC group design 


Highlights from the Field J 


Developing and leveraging industry partnerships 


A consortium of four educational institutions—referred to as “hubs”—in three states developed and 
implemented a TAACCCT Round 2-funded initiative aimed at expanding the breadth and effectiveness of the 
training options and career pathways through which participants could work toward careers in the shale oil 
and gas industry. 


Central to development and implementation of the training options was sustained industry partnerships. 
Over the course of the grant, the ShaleNET consortium engaged more than sixty unique employers and 
industry organizations. 


Each hub’s industry engagement was maintained by designated Hub Directors and Career Counselors, and 
supported by advisory groups and quarterly ShaleNET hub meetings. These meetings provided industry 
representatives with opportunities to learn more about ShaleNET, to give their input about program design 
and activities, and to explore how ShaleNET training programs could help them meet their workforce needs. 
This included valuable input and feedback about core competencies and training components in the training 
programs. For example, industry representatives were instrumental in helping ShaleNET hubs set up their 
well site equipment, maintain it, and use it to provide meaningful hands-on learning. 


In addition, employers also provided ShaleNET students opportunities for work-based learning, including site 
visits and internships. These opportunities allowed students to get into the field and interact with industry 
equipment and workers. 


Finally, the ShaleNET partners provided leveraged resources by funding scholarships and donating 
equipment and supplies. These included pumps, control valves, controllers, hard hats, safety goggles, 
and gloves. 


(Dunham et al., 2015) 


WWC 2021007 Designing and Delivering Career Pathways at Community Colleges | Recommendation 5 | 46 


Recommendation 5 


Table 7. Results of Meta-analysis for Recommendation 5 


Statistically Statistically 

Yefaliicer-laiar-lae| Sle Talnicer-talar- ae} 
Outcome Domain positive Indeterminate negative 
Short-term employment @ 
Medium-term employment @ 
Short-term earnings e 
Medium-term earnings @ 
Note: Across the 11 studies, there were findings in four outcome domains even though all six labor market outcome domains 
(see Box 1) were relevant for this recommendation. None of the 11 studies included findings that meet WWC standards in the 
following outcome domains: (1) long-term employment; or (2) long-term earnings. 


standards without reservations, and the other 
four studies! meet WWC group design standards 
with reservations. An additional four studies! 
include employment-focused partnerships but do 
not report any labor market outcomes. 


Results of a meta-analysis indicate a statistically 
significant positive effect of these interventions 
on short-term employment and short-term 
earnings. Effects on medium-term employment 
were indeterminate. Educational outcomes 
were not considered for this recommendation. 
The average effect size is not statistically 
significant and negative for any of the three 
labor market outcome domains relevant to this 
recommendation. In 10 of the 11 studies used 

to support Recommendation 5, employment- 
focused partnerships are a secondary component 
of the tested intervention. See Appendix C for 
a detailed rationale for the level of evidence 
assignment for Recommendation 5. 


It is important to acknowledge the limitations in 
the current research available on employment- 
focused partnerships. First, the majority of the 
interventions in the studies referenced in this 
practice guide engaged external employment- 
focused partners such as employers, workforce 
boards, industry associations, and American 


Job Centers. However, the studies did not 
directly test employment-focused partnerships 
as an intervention; rather, these external 
partnerships were a component of the larger 
intervention. Second, available research often 
includes varying information on the strength 
of external partnerships, such as the quality of 
the relationships or how deeply partnership 
members engage and collaborate with one 
another. Finally, because employers may have 
multiple partnerships with multiple colleges, 
students in study comparison groups could 
receive and benefit from some level of services 
provided by these partners outside of a specific 
intervention. This loss of contrast between 
treatment and comparison groups can make it 
difficult to tease out differences in labor market 
outcomes caused by the interventions. 


This section describes strategies, examples, 

and tools that can support community college 
administrators, faculty, and staff in their efforts 
to build, sustain, and leverage employment- 
focused partnerships. All figures and mentions 
of specific community colleges in 
Recommendation 5 are offered as examples 
only and should not be read as endorsements of 
specific products or approaches. 
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Recommendation 5 


How to Carry Out the 1. Identify ways to get employers strategically 


Recommendation engaged in each aspect of planning and 
implementing career pathways. 


The guidance below is informed by the studies 
that support the recommendation as well as 
the expert panel’s knowledge of strategies and 
resources available to help implement 

the recommendation. 


When employers and industry partners 

are continuously engaged in all aspects of 
career pathways design and implementation, 
students are better prepared when they 

take an exit along the career pathway for 


Figure 5.1. Examples of employer engagement 


Assistance * Recruit students 
¢ Provide mentors 
¢ Provide work-based learning experiences or in-service activities for instructors 
¢ Provide industry-based training to instructors 
¢ Support instructors’ memberships/participation in trade associations 


¢ Provide work-based learning experiences for students, such as simulations, labs, field trips, 
internships, clinicals, paid and unpaid work experiences 


¢ Teach a module or topic 

¢ Advise on resource needs, including equipment, labs, staff qualifications, facilities 
¢ Assist in the acquisition of equipment and supplies 

¢ Assist with placement of program completers 

¢ Provide resources, such as equipment, materials, facilities 

¢ Share libraries of visual aids, books, and magazines 


Assessment — * Review curriculum, including objectives, structure and length, curriculum frameworks and 
and Counsel course descriptions, assessments, industry credentials, safety, course materials 


¢ Advise on academic, technical, and employability standards 


¢ Modify existing programs and/or identify new or emerging fields for which programs should 
be developed 


¢ Review student outcomes, such as completion rates, placement rates, and state licensing 
exam outcomes 


¢ Assist in surveys of local labor market needs 
¢ Assist in identifying equipment needs 


¢ Assess the equipment and facilities available and make recommendations as needed 


Source: https:/Awww.advancingcredentials.org/toolkit/index.php 
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employment. Community colleges can engage 
employment-focused partners in a variety of 
ways and at various stages of career pathways 
planning and delivery, as well as when they 
are supporting students with job searches 
and placement. As shown in Figure 5.1, 
employer engagement can be in the form of 
curriculum planning and review, assessment 
of local labor market information, assistance 
with student recruitment, provision 

of mentorship or work-based learning 
opportunities, as well as active promotion 

of the career pathway program. 


The level of engagement of employment- 
focused partners may vary depending on the 
partner type and local or regional context of 
a given career pathway. For example, newly 
created pathway programs for emerging 
industries may initially involve few partners 
in several aspects of planning, but eventually 
involve more partners who are involved 

in different aspects of ongoing program 
implementation and improvement efforts. 
Jobs for the Future’s A Resource Guide to 
Engaging Employers'” includes a “ladder of 
employer engagement” that depicts how 
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employers can play roles that range from 
advising, capacity building, co-designing, 
and convening to leading (see Figure 5.2). 
The expert panel recommends that the 
specific stages and activities be modified 

to local needs and circumstances, and that 
community colleges remember the best way 
to increase employer engagement is to take 
the time to build meaningful relationships 
with employers. 


Employers and industry partners can serve 
on advisory committees as strategic partners 
for the program.’ In order for an advisory 
commnittee to be effective, its members must 
be engaged strategically at various levels 

of program design and implementation.'° 
When given the opportunity to provide 
meaningful feedback and guidance during the 
college’s planning, members can indirectly 
shape students’ experiences and outcomes 
in career pathways."° The Association for 
Career & Technical Education offers several 
illustrative tools and examples to promote 
strategic engagement with industry partners 
in advisory boards or committees." 


Figure 5.2. Sample Ladder of Employer Engagement 


NV (-1 | Level IV | =\V7-) AVA 


Level | | A=NV (=) | 


Key employer Advising Capacity-building Co-designing Convening Leading 
role 
Stage of Initial contact / Establishing trust | Working | Trusted provider Full strategic 
relationship newrelationship — and credibility relationship and collaborator — partner 
Activity Discuss hiring | Job site tours; Curriculum College-employer Multi-employer 
examples needs, skills, speakers; mock and pathway sectoral / multi-college 
competencies; interviews; development; partnerships partnerships 
advise on internships; needs adjunct faculty 
curricula; assessment; and preceptors 


contract training; loan/donate 
hire graduates equipment; 
recruiting 


Source: Wilson, R. (2015). A Resource Guide to Engaging Employers. Boston: Jobs for the Future. (httos://jfforg-prod-new. 
s3.amazonaws.com/media/documents/A-Resource-Guide-to-Employer-Engagement-011315.pdf) 
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Employment-focused partnerships 

can support the design and delivery of 
instructional content and activities that 

are relevant and engaging to students. 
Employers, for example, can offer students 
valuable exposure to work environments 
through various work-based learning 
opportunities, such as hosting experiential 
field trips, internships, clinical placements, or 
apprenticeships."” Students can also benefit 
by learning from industry experts serving as 
instructors or guest lecturers. Instructors can 
also re-engage former students by inviting 
them to serve as guest speakers or mentors 
for current students. 


Employers and industry partners can also 
play a critical role in delivering job search 
and placement supports to career pathways 
students. Career fairs and hiring events 
provide students with opportunities to meet 
and network with potential employers."* 
Employers who are actively aware of or 
engaged in the design and delivery of a career 
pathway can help prepare students through 
mock interviews. The partner may even 
agree to interview or hire a certain portion 
of students who complete the pathway’s 
programs and credentials. 


Conduct an inventory to identify current 
and potential employment-focused 
partnerships. 


In order to best leverage employment-focused 
partnerships to support students’ labor 
market success, community colleges should 
conduct an inventory to identify current and 
potential partners and think critically about 
how to engage with them. 


The panel suggests community colleges 
consider the following in mapping out 
existing and potential employment-focused 
partnerships: 


¢ Who is the employer point of contact? 


¢ What is the current engagement of the 
employer? 
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* Does the employer engage with multiple 
departments or offices at the college? 


¢ How would you like the employer to 
engage with the college in the future? 


¢ Is there a formal agreement (e.g., 
Memorandum of Understanding) in place 
with the partner? 


To identify new employment-focused 
partners, colleges can first connect with local 
workforce boards. They can also reach out to 
local, state, or regional industry associations. 
National industry associations and CTE 
accreditation bodies can also be leveraged. 
These organizations may offer job placement 
data that may provide information about 
prospective employment-focused partners. If 
a career pathway has been established for a 
few years, a college should consider accessing 
and analyzing data about where its graduates 
are employed, and engage any employers not 
already partnering with the college. 


Colleges may consider conducting a “self- 
check” to identify opportunities to strengthen 
or deepen partnerships with leaders in 
specific occupational sectors that are relevant 
to the career pathway(s) offered by the 
college. The Employer Engagement Toolkit 
created through the ECMC Foundation- 
funded Advancing Credentials Through Career 
Pathways initiative includes an institutional 
checklist for employer engagement, featured 
here in Figure 5.3."* The checklist assesses 
the level of engagement with local employers 
in terms of collaboration around job fairs, 
mentorship and work-based learning, and 
post-graduation hiring, among other activities. 


The National Council for Workforce 
Education suggests colleges should develop 
an employer engagement plan that includes 
an environmental scan of internal and 
external perceptions of employer engagement 
and its effectiveness; college goals; an 
employer checklist and database; and a 
communication plan that covers internal and 
external communications.'® 
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Figure 5.3. Checklist for employer engagement 


This self-check is for your information only. It may be completed by a variety of college staff followed by a 
discussion of options for gathering and gaining access to information. 


Gathering Input 
1. Do employers offer internships geared specifically to your program’s students? 


OO Yes CJ No [J Don’t Know 

2. Are your students offered job shadowing opportunities with employers in your area? 
C] Yes 1 No () Don’t Know 

3. Do you include guest speakers or industry tours as part of your program? 
0 Yes O1 No [] Don’t Know 

4. Do you have an up-to-date list of guest speakers/industry tours for your program courses? 
1 Yes [1 No CL] Don’t Know 

5. Have you continued to grow the list and add new speakers/tours as new companies come to your area? 
1 Yes [1 No C1 Don’t Know 

6. Do employers provide mentorship for your students throughout their program? 
CL] Yes O1 No ( Don’t Know 

7. Does your college hold career fairs? 
C1] Yes O01 No [J Don’t Know 

8. If so, are they sector specific? 
O1 Yes OI No (] Don’t Know 

9. Has the number of employers participating in career fairs grown over the past 5 years? 
O1 Yes 01 No [] Don’t Know 

10. Are both large and small employers represented? 
O1 Yes O1 No (] Don’t Know 

11. Do you include employers in mock interviews to help students prepare for job searches? 
1 Yes ] No [J Don’t Know 

12. Do you know the employment rate of your graduates for each program your college offers? 
LI Yes LJ] No 

13. Do you know which employers hire your graduates year after year? 
O Yes CJ No 

14. Do you know which employers don’t hire your graduates and the reason(s) why? 
O Yes [No 

15. Do employers send their current employees to your college for continuing education and skills upgrades? 
O Yes O01 No LJ Don’t Know 


16. Do employers suggest/recommend your program to job candidates who lack needed skills? 
O Yes [J No LJ Don’t Know 


Source: https://www.advancingcredentials.org/toolkit/index.php 
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3. Designate staff time and resources to 


build and sustain employment-focused 
partnerships. 


Engagement with employment-focused 
partners allows college administrators and 
staff opportunities to better understand 
employer and industry expectations and 
labor market needs. When considering the 
benefits of such partnerships, administrators 
should also consider how they will invest 
resources and allocate staff time to build 

and then maintain the partnerships. For 
each career pathway, it will be important 

to determine how many staff will need to 
dedicate part or all of their time to engaging 
employers and industry partners. When 
feasible, employment-focused partnerships 
can be facilitated by providing physical space 
on campus for employer engagement and for 
employers to engage directly with students 
and faculty. 


Involving employers and industry partners in 
the planning and design of career pathways 
may improve job placement rates when 
students complete their pathway studies."° 
When colleges communicate openly and 
frequently with their partners, they are 
better equipped to respond to feedback and 
concerns with appropriate adjustments to 
the program or curriculum to better prepare 
students for employment. 


To ensure continuous engagement and 
communication, the expert panel believes 

it is important for colleges to establish a 
timeline for formal and informal check-ins 
with their employment-focused partners. In 
emerging industries, this timeline may entail 
more frequent communications. 


Further, the expert panel suggests 
administrators, faculty, and staff within— 
and possibly even across—career pathways 
should develop communication plans so 
they are clear about who is reaching out 
to which employers with specific requests. 
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This may involve setting up a data system to 
organize contacts and coordinate and track 
communications. Investing in this type of 
data system can both streamline and improve 
communications with employers, who may 
otherwise receive multiple, perhaps even 
conflicting, requests from a college. 


Collaborate with employers and industry 
partners to provide students hands-on 
learning opportunities that are relevant 
to occupations along their chosen 

career pathway. 


When career pathways provide students 
with hands-on exposure to occupations 
they are pursuing—or might pursue in 

the future—students can develop realistic 
expectations about those occupations. 
Project-based learning, high-fidelity 
simulations, and experiential field trips are 
strategies that instructors can use to align 
course content and delivery with students’ 
career interests.'” Internships, work-based 
learning opportunities, clinical placements, 
and apprenticeships allow students to gain 
employment experience as they progress 
along a career pathway. Several examples of 
employer engagement, each from Wisconsin’s 
Making the Future initiative, are featured in 
Figure 5.4. 


In 2017, some 50 percent of full-time 
students and 78 percent of part-time students 
attending two-year postsecondary institutions 
were employed."* Students are often 
employed while pursuing postsecondary 
studies so they can balance their educational, 
personal, and financial obligations. To the 
extent possible, faculty and staff should work 
with students to seek employment that is 
aligned to students’ career interests, such as 
employment in their targeted industry, and 
conducive to students’ educational goals, for 
example on campus or on a flexible work 
schedule. They also should find ways to 
connect students’ work experience to their 
education through reflection activities. 
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Figure 5.4. Examples of employer engagement 


Moraine Park Technical College’s welding boot 
camp represents a unique expansion of the 
employer’s role in education and training at the 
college. The welding boot camp includes a 72- 
hour paid internship for all participants with a local 
employer who selects the interns after reviewing 
resumes and test scores (math and reading, as well 
as a mechanical assessment), which then provides 
entry to the welding boot camp at the college. 
While the college initially had to recruit employers 
to participate and offer internships, it now has more 
employers wanting to participate than available 
slots. This process has resulted in an overall job 
placement rate of 86% within three months of 

boot camp completion. The internship program 
attracted students who would never otherwise have 
considered college, but the strong connection with 
employers and the possibility of a job motivated 
participation. As one student noted, “my [internship] 
supervisor tells me | have potential as a welder and 
| should have no problem getting employed. This 

is very encouraging. There aren’t many people or 
places willing to teach a skill like this to a beginner.” 


Both Blackhawk Technical College and Chippewa 
Valley Technical College developed new internship 
opportunities for manufacturing students. 
Blackhawk used an internship coordinator to 
develop opportunities for students in the industrial 
maintenance program, while CVTC assigned 
faculty in the machine tool program to manage its 
program. Both colleges identified a course within 
their respective manufacturing programs where 

an internship could be taken as an alternative to 

a program course. And both colleges established 
formal protocols to guide, manage, and assess the 
internship activity. 


Another aspect of expanded employer partnerships 
involves preparing students for employment and 
providing priority consideration in hiring decisions. 
Southwest Technical College engaged local 
employers in the development of the Gold Collar 
Certificate training, and successfully worked with 

a number of employers to provide student tours 

of their facilities, help prepare students for job 
interviews, and offer priority hiring for students who 
complete the bridge. Notably, the 3M Company 
adapted its hiring profile to ask if students had 
earned the GCC, and publicly indicated that 
completers receive priority consideration. 


Source: Price et al. (2016) 
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Colleges also can set up “learn-and-earn” 
models, such as internships, cooperative 
education, pre-apprenticeships and 
apprenticeships, federal work-study, or 
practicums, that allow students greater 
access to work experiences related to their 
career-oriented instruction and financial 
assistance."° Year Up, a national workforce 
training program for young adults living in 
urban areas, is an example of a program that 
applies the “learn-and-earn” model. It has 
established relationships with local businesses 
so that Year Up students are placed in paid 
internships in their targeted industry during 
the second half of the program.”° 


In cases where students are pursuing 
stackable credentials, they may reach a 
point along their career pathway where 
they would benefit from gaining workplace 
experience before pursuing further 
education, credentials, or employment. This 
is because stackable credentials are aligned 
to industry certifications and represent “exit 
points” at which students have reached 

a designated milestone. In other words, 

a student earning a certification may be 
prepared for the first level of employment 
and can pick up where they left off when 
they return to continue their training. As 
appropriate, faculty and staff can advise 
students to take advantage of employment 
opportunities at transition points along the 
career pathway. When doing so, they should 
consider how and when they will encourage 
students to return for additional education 
and training, and what kinds of supports 
students will need to successfully re-enter 
their career pathway program.”! 


Regularly review the employment- 
focused partnership inventory to assess 
whether or not the right partners are 
involved to help advance students’ labor 
market outcomes. 


Community college administrators and 
staff can use partnership inventories to 
identify gaps that may need to be filled 
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by strategically engaging employers and 
industry partners. If implementing a strategic 
plan for employer engagement, they can 
systematically track engagement activities 
and assess whether the efforts are benefitting 
students in their career pathways. 


Ongoing communications with current and 
former students and employers is important 
for program improvement. Colleges should 
consider developing and implementing 
mechanisms to allow employers the 
opportunity to provide feedback on their 
pathways programs and student hires, 
including the extent to which colleges are 
preparing workers with the right skills.” 
Mechanisms may include phone calls, emails, 
surveys, and face-to-face meetings. Colleges 
can also share program and student successes 
with employers, along with information 
about improvements they are making based 
on the feedback employers and industry 
partners provided. 


Students and employers can provide 
feedback on the benefits and challenges 

of both the curriculum and its hands- 

on learning opportunities, potentially 
informing improvements to the design 

and implementation of career pathways 
programs. Communication with students 
and employers can take place formally 

and informally.’? Semi-annual partner 
roundtables provide one such venue, in 

that these roundtables provide colleges and 
partners a formal opportunity to discuss the 
career pathways model, solicit feedback, and 
present results, with the expectation that the 
college will implement the feedback into the 
model or pathway design.'* 
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Potential Obstacles and the 
Panel’s Advice 


OBSTACLE: Implementing hands-on learning 
opportunities for career pathways students requires 
a high level of coordination between college and 
employer, which may be costly and time consuming. 


PANEL’S ADVICE: The demand for hands- 

on learning opportunities—whether offered on 
campus in the classroom or lab or offered at 

a job site—requires strong, well-coordinated 
partnerships between employer and campus staff. 
These well-coordinated partnerships are often 
forged over a period of time, giving both college 
and employer a clear understanding of the benefits 
to each of the partnership. Where possible, 
colleges can dedicate career counselors or other 
staff to helping make and maintain connections 
between them and employers. Findings from an 
evaluation of the Accelerating Connections to 
Employment initiative suggest it may be necessary 
to hire both a career navigator to guide students 
along their career pathway and a job developer 

to help students secure hands-on learning 
opportunities and post-program employment.” 


Planning tools such as the one featured in 
Figure 5.5 can help colleges determine the best 
opportunities to integrate work-based learning 
into their career pathways offerings. Such tools 
can also help colleges remain attentive to the 
logistics of partnering with employers, such 

as any pre-training requirements, supervision 
expectations, paperwork, and liability when 
students are on a job site. The U.S. Department 
of Education’s National Center for Innovation 

in Career and Technical Education prepared a 
Work-Based Learning Toolkit that includes several 
strategies for and examples of engaging employers 
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Figure 5.5. Planning tool for work-based learning 


Employers Employer Staff 
Type Of Work-Based Experience Interested Needs Timeline | Responsible 


Summarize 
List what the Target date 
employers interested to begin Action items 
who are employers work- Lead to begin 
open to require from based within your work-based 
Internship, OJT, etc. hosting you tohost experience organization opportunities 


Paid / Unpaid Internship 

Employers define a brief job or project 
that requires the intern to use and 
improve specific skills under the 
guidance of a supervisor. Learning 
goals and achievement goals are set 
by the employer and the experience is 
formally incorporated into the training 
curriculum. When the internships are 
paid stipends are set and paid by the 
employer unless funds are available 
from the training program. Remember 
_ that unpaid internships must comply 
with the Fair Labor Standards Act. 


-Temporary Employment 

Employers “hire” the employee for a 
set duration to perform duties similar 
to those in the jobs participants are 
training for. Often, this occurs through 
a temporary employment agency. 
Temporary employment can be more 
easily incorporated into a training 
program if work is part-time so that 

_ participants can spend roughly half 

_ their program time working and half in 


training and skills classes. 


WIOA On-the Job Training 
Employers provide short-term, 
structured training as part of their 
workers’ job responsibilities. The public 
workforce system reimburses the 
employer, generally up to 50 percent 
of the participant’s wages, for the costs 
of training and supervision. This can 
help incentivize businesses to hire 
new workers who still require some 
occupational training to succeed in their 
new job. 


Other Ask employers for other options 
such as apprenticeships that work 

for them for trainees to gain real job 
experience at their company during 
training. 


Source: Adapted from Job for the Future’s Employer Engagement Toolkit, Tool 4.5 (https://ifforg-prod-new.s3.amazonaws.com/ 
media/documents/4.5-Work-Based-Experience-0924 15. pdf) 
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in work-based learning. Topics include addressing 
employers’ liability concerns, demonstrating to 
employers that participants have the necessary 
skills for the workplace, offering a range of work- 
based learning activities to allow employers 

to participate at different levels, and reaching 
employers through business intermediaries.'”° 


OBSTACLE: Employers may not understand the 
value of partnering with community colleges to help 
train workers who are not (yet) their employees. 


PANEL’S ADVICE: Partnering with community 
colleges can provide employers access to a 

pool of talent whose skills are valuable to the 
employer and the industry; it can also ultimately 
reduce employers’ costs to identify, recruit, 

and train new employees (or retrain current 
ones).!” Still, some employers may be hesitant to 
partner with colleges. Colleges can proactively 
share data with employers to help them see the 
value of investing time and resources into career 
pathways partnerships.”° They can share student 
outcomes data, such as program completion 
rates, credential or degree attainment rates, 

and job placement and retention rates. They 
can also share anecdotal evidence collected 
through formal or informal communication with 
former students and their current employers 
and with existing employment-focused partners. 
Colleges can also explore employers’ interest 

in offering hands-on learning opportunities for 
potential employee candidates. Finally, they 

can engage industry associations and advocacy 
groups to help secure employers’ buy-in to earn- 
and-learn concepts. 


When appropriate, a college may need to 
demonstrate the return on investment that 
employment-focused partners can expect from 
partnering with it. Community colleges can 
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consider the following steps for calculating 
that return: 


1. Listing investments of time, money, and 
resources of the employment-focused 
partner. 


2. Listing benefits that will accrue to the 
employer as a result of its investment. 


3. Translating both costs and benefits into 
dollar values. 


4. Demonstrating the training program was 
responsible for generating the benefit, such 
as time savings, productivity increases, or 
personnel savings.'° 


When planning work-based experiences, colleges 
should be mindful that some employers may be 
hesitant to support or host these opportunities. 
Some employers may have concerns 

about Unemployment Insurance, Workers 
Compensation, and/or liability. Where possible, 
colleges should work with legal counsel from 
their institution and with their state’s attorney 
general to explore ways to reduce or share such 
risks with employers. 


Another way to secure employer support is 
to engage employers in conversations about 
the characteristics of an ideal employee 
candidate. Such conversations can help the 
college determine how to evaluate the quality 
and preparedness for future employment of 
students completing the college’s programs. 
Colleges can tailor their curriculum, instruction, 
and assessments to evaluate technical or 
employability skills and to prepare students 
for specific industry certification exams that 
employers value, and often require. 
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A 


academic program review is a process that evaluates the status, effectiveness, and progress of 
academic programs and helps identify the future direction, needs, and priorities of those programs. 
Both student and community needs may be taken into consideration when conducting academic 
program review for career pathways programs. 


acceleration strategy encompasses varying ways of shortening the time required for completing basic 
skills training and occupational coursework. These strategies include compressed courses, co-requisite 
courses, integrated basic skills training, and awarding credit for prior learning, among others. 


administrator refers to any person who is in a position of authority or who manages people, 
practices, and policies at a postsecondary institution of education. 


adult education programs provide academic instruction and education services below the 
postsecondary level that increase an individual’s ability to: read, write, and speak in English and 
perform mathematics or other activities necessary for the attainment of a secondary school diploma or 
its recognized equivalent; transition to postsecondary education and training; and obtain employment. 


adult learning theory is the concept or study of how adults learn and how it differs from children. 
Adult learning theories provide insight into how adults learn, and can help instructors be more 
effective in their practice and more responsive to the needs of the learners they serve. 


advisory committee refers to a group of local employers and other stakeholders involved in 
providing strategic guidance on decisions related to the implementation, improvement, and evaluation 
of career and technical education programs. 


American Job Centers provide free assistance to job seekers for a variety of career and employment- 
related needs. 


apprenticeship is an industry-driven, high-quality career pathway where employers can develop 
and prepare their future workforce, and individuals can obtain paid work experience, classroom 
instruction, and a portable credential. 


articulation agreement, as defined in the Strengthening Career and Technical Education for the 
21st Century Act (Perkins V), means a written commitment-— (A) that is agreed upon at the State level 
or approved annually by the lead administrators of-— (i) a secondary institution and a postsecondary 
educational institution; or (ii) a subbaccalaureate degree granting postsecondary educational 
institution and a baccalaureate degree granting postsecondary educational institution; and (B) to 

a program that is— (i) designed to provide students with a nonduplicative sequence of progressive 
achievement leading to technical skill proficiency, a credential, a certificate, or a degree; and (ii) 
linked through credit transfer agreements between the 2 institutions described in clause (i) or (ii) of 
subparagraph (A) (as the case may be). 
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B 


basic skills refers to foundational/prerequisite skills—typically in reading, writing, and math— 
necessary for students to succeed in career and technical education coursework. 


block scheduling refers to scheduling courses for longer periods of time, resulting in fewer but longer 
class periods. 


braided funding is the weaving together of various federal, state, and private funding streams and 
resources. 


bridge program provides basic (or prerequisite) skills training required for students to pursue 
occupational coursework. Bridge programs act as pre-college or pre-pathway onramps to occupational 
training, and are sometimes referred to as “pre-pathway bridge programs.” 


C 


career and technical education (CTE), as defined in the Strengthening Career and Technical 
Education for the 21st Century Act (Perkins V), is organized educational activities that (A) offer a 
sequence of courses that— (i) provides individuals with rigorous academic content and relevant 
technical knowledge and skills needed to prepare for further education and careers in current or 
emerging professions, which may include high-skill, high-wage, or in-demand industry sectors or 
occupations, which shall be, at the secondary level, aligned with the challenging State academic 
standards adopted by a State (ii) provides technical skill proficiency or a recognized postsecondary 
credential, which may include an industry-recognized credential, a certificate, or an associate degree; 
and (iii) may include prerequisite courses (other than a remedial course) that meet the requirements 
of this subparagraph; (B) include competency-based, work-based, or other applied learning that 
supports the development of academic knowledge, higher-order reasoning and problem-solving skills, 
work attitudes, employability skills, technical skills, and occupation-specific skills, and knowledge of 
all aspects of an industry, including entrepreneurship, of an individual; (C) to the extent practicable, 
coordinate between secondary and postsecondary education programs through programs of study, 
which may include coordination through articulation agreements, early college high school programs, 
dual or concurrent enrollment program opportunities, or other credit transfer agreements that 
provide postsecondary credit or advanced standing; and (D) may include career exploration at the 
high school level or as early as the middle grades. 


career lattices (also referred to as career ladders) help people visualize the potential job 

pathways available as they progress through a career by graphically depicting the occupations and 
developmental experiences within an industry. They consist of a group of related jobs that make up 
a career, and often include a pictorial representation of job progression in a career as well as detailed 
descriptions of the jobs and the experiences that facilitate movement between jobs. 


career navigator is a student support staff member who primarily assists students in clarifying and 
linking academic, personal, and professional goals in relation to a particular occupation. 


competency-based education is a program of study based on the mastery of specific information and 
skills tied to application in the workforce. Competency-based programs award credentials based on 
demonstrated ability rather than participation in course instruction. 
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contextualized instruction refers to instructional strategies that make explicit connections between 
learning of basic skills and occupational content. 


co-requisite courses allow students to enroll in basic skills training and occupational courses 
simultaneously. Sometimes referred to as dual/concurrent enrollment. 


credit for prior learning refers to awarding credit for the knowledge and skills a student gained 
through past work and life experience. 


D 


degree refers to an award or title conferred by a postsecondary education institution upon a student 
for the completion of a program of study. A type of credential. 


does not meet What Works Clearinghouse group design standards identifies a group design study 
with a low level of causal evidence. This is the rating given to studies with causal research designs that 
were not implemented rigorously enough to conclude with confidence that the intervention caused the 
observed changes in outcomes. 


E 


employment-focused partnerships, for the purpose of this guide, are defined as partnerships 
between community colleges, employers, and industry partners (workforce boards, industry 
associations, and American Job Centers, for example) that focus on supporting the educational and 
labor market outcomes of students who participate in career pathways. 


F 


faculty refers to any individual who teaches full- or part-time at the postsecondary level. This includes 
instructors and adjunct professors; it can also include graduate students and other staff responsible for 
teaching courses. 


family sustaining employment is employment that pays a family-sustaining wage, offers benefits 
including paid sick leave, and offers career pathways that provide opportunities for wage and career 
advancement. 


flexible instruction refers to different ways of offering coursework at times, places, and in formats 
that accommodate and facilitate participation by students balancing their education with work and 
family obligations. These ways include evening and weekend courses, block scheduling, self-paced 
learning, and online learning. 


flipped classroom is an instructional strategy and a type of hybrid learning focused on student 
engagement and active learning that requires students to learn and review content prior to classroom 
instruction. This gives the instructor a better opportunity to deal with mixed levels, student 
difficulties, and differentiated learning styles during in-class time. 
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G 


group design standards are the set of standards used by the What Works Clearinghouse to review 
studies. These standards are used to evaluate the strength of the evidence of the effectiveness of 
educational interventions. Studies are given a rating of Meets WWC Group Design Standards without 
Reservations, Meets WWC Group Design Standards with Reservations, or Does Not Meet WWC Group 
Design Standards. 


H 


hybrid learning combines face-to-face classroom instruction with online learning, reducing 
classroom contact hours. Sometimes referred to as blended learning. 


I 


integrated basic skills instruction refers to a curriculum design where basic skills instruction and 
occupational content instruction are delivered simultaneously, often by a team comprising a basic 
skills instructor and an occupational content instructor. 


internship is a work-based learning opportunity provided by partnerships between educational 
institutions and employers to give students practical experiences in their field of study. 


L 


labor market information refers to data and other information from employers that can be used to 
understand labor market conditions and trends in a particular industry and geographical area. 


license refers to a credential granted by a federal, state, or local government agency in accordance 
with preestablished licensure requirements. A license awards an individual legal permission to 
perform certain regulated tasks or occupations. 


M 


meets What Works Clearinghouse group design standards with reservations is the middle 
possible rating for a group design study reviewed by the WWC. Studies receiving this rating provide 


a lower degree of confidence that an observed effect was caused by the intervention. Randomized 
controlled trials that are not as well implemented or have problems with attrition, along with strong 
quasi-experimental designs, may receive this rating. 


meets What Works Clearinghouse group design standards without reservations is the highest 
possible rating for a group design study reviewed by the WWC. Studies receiving this rating provide 
the highest degree of confidence that an observed effect was caused by the intervention. Only well- 
implemented randomized controlled trials that do not have problems with attrition may receive this 
highest rating. 


meta-analysis is a quantitative statistical analysis of several separate but similar experiments or 
studies in order to test the pooled data for statistical significance. 
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N 


non-degree credential include: certificates issued in recognition of the completion of a 
postsecondary course of study, which can be either for-credit or non-credit; industry certifications 
awarded by an industry body, business, or trade association for demonstration of skills, typically via 
examination, based on industry standards; occupational or professional licenses awarded by a state 
or federal agency for demonstration of skills in a specific occupation; and apprenticeship credentials 
awarded after completion of a structured educational and workplace program based on industry and 
occupational standards. Emerging non-degree credentials include badging and microcredentialing. 
These credentials can be earned on the path to an Associates of Applied Associates degree. 


O 


outcome domain is a group of closely-related outcomes. A domain is the organizing construct for a 
set of related outcomes through which studies claim effectiveness. For example, the WWC alphabetics 
domain within the Literacy topic includes the following outcomes: phonemic and phonological 
awareness, letter identification, print awareness, and phonics. In intervention reports and practice 
guides, the WWC assesses the rigor of evidence on the effectiveness of interventions within each 
domain identified in the review protocol. The intervention rating, improvement index, and extent of 
evidence are determined at the domain level. 


P 


prior learning assessment (PLA) is a method of determining and awarding credit for the knowledge 
and skills a student gained through past work and life experience. 


proactive advising refers to an advising model where advisers proactively and continuously engage 
students, with the goal of developing a caring and beneficial relationship that leads to students’ 
increased academic motivation and persistence. Sometimes referred to as intrusive advising. 


program of study is a structured sequence of academic and career and technical courses leading to a 
postsecondary-level credential. 


S 


self-paced learning is any kind of instruction that proceeds based on the student’s response and level 
of competence. 


stackable credentials are credentials that can be accumulated over time and move a student/worker 
along a career pathway or up a career ladder. 


Ww 


work-based learning refers to varying approaches to providing students with employment-relevant 
experiences as part of the training program. These approaches include internships, apprenticeships, 
visits to local employers, clinical placements, and job shadowing. 
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Appendix A: Postscript From the Institute of Education Sciences 


What Is a Practice Guide? 


The Institute of Education Sciences (IES) publishes practice guides to share coherent expert guidance 
addressing a particular educational challenge. Each recommendation in the practice guides is explicitly 
connected to supporting evidence from studies that meet What Works Clearinghouse™ (WWC) standards. 


How Are Practice Guides Developed? 


To produce a practice guide, IES first selects a topic. Topic selection is informed by inquiries on the 
WWC website and requests sent to the WWC Help Desk, a limited literature search, and an assessment 
of the topic’s evidence base. Next, working with a WWC contractor, IES selects a panel chair who has 
a national reputation and expertise in the topic, as well as additional panelists to co-author the guide. 
Panelists are selected based on their expertise in the topic area and the belief that they can work 
together to develop relevant, evidence-based recommendations. Panels include at least two current 
practitioners with expertise in the topic. 


Relevant studies are identified through panel recommendations and a systematic literature search. 
These studies are then reviewed against the WWC design standards by certified reviewers who assess 
the internal validity of each study.”° The panel synthesizes the evidence into recommendations. WWC 
staff summarize the research and draft the practice guide. 


IES practice guides are then subjected to external peer review. This review is independent of the 
panel and the IES and WWC staff that supported the development of the guide. A critical task of the 
peer reviewers is to determine whether the evidence cited in support of particular recommendations 
is up-to-date and that studies of similar or better quality that point in a different direction have not 
been overlooked. Peer reviewers also evaluate whether the level of evidence category assigned to each 
recommendation is appropriate. WWC staff revise the guide to address concerns identified by the 
external peer reviewers and IES. 


Levels of Evidence for What Works Clearinghouse Practice Guides 


The level of evidence represents the quality and quantity of existing research supporting each 
recommendation. The WWC and the panel assign each recommendation one of the following three 
levels of evidence: strong evidence, moderate evidence, and minimal evidence. 


A strong level of evidence rating refers to consistent evidence that the recommended strategies, 
programs, or practices improve relevant outcomes for a diverse population of students."! In other 
words, this level of evidence indicates that there is strong causal and generalizable evidence to support 
the panel’s recommendation. 


A moderate level of evidence rating refers either to evidence from studies that allow strong causal 
conclusions but cannot be generalized with assurance to the population on which a recommendation 
is focused (perhaps because the findings have not been widely replicated), or to evidence from studies 
that are generalizable but have some causal ambiguity. 
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A minimal level of evidence rating suggests that the panel and the WWC cannot point to a body of 
evidence that demonstrates the practice’s positive effect on student outcomes. In some cases, this 
simply means that the recommended practices would be difficult to study in a rigorous, experimental 
or quasi-experimental fashion; in other cases, it means that researchers have not yet studied this 
practice, or that there is weak or conflicting evidence of effectiveness. A minimal evidence rating 
does not indicate that the panel views the recommendation as any less important than other 
recommendations with a strong or moderate evidence rating. 


To determine these levels of evidence, the WWC along with the panelists first conducts a careful 
review of the studies supporting each recommendation. For each recommendation, they examine the 
entire evidence base, taking into account the following considerations: 


¢ Relevance of studies for representing the range of participants, settings, and comparisons on which 
the recommendation is focused. 


¢ Whether findings from the studies can be attributed to the recommended practice. 


¢ The weighted mean effect size from the fixed-effect meta-analysis for each relevant outcome 
domain." 


¢ The extent of evidence meeting WWC standards.™ 
¢ How well the studies represent the range of participants and settings relevant to the recommendation. 
¢ The panel’s confidence in the effectiveness of the recommended practice. 


In developing the levels of evidence, the panel and the WWC consider each of the criteria in Table A.1. 
The level of evidence rating for a recommendation is determined based on the findings for each of the 
criteria. For a recommendation to get a strong rating, the research must be rated as strong on each 
criterion. If at least one criterion receives a rating of moderate and none receives a rating of minimal, 
then the level of evidence for the recommendation is determined to be moderate. If one or more 
criteria receive a rating of minimal, then the level of evidence for the recommendation is determined 
to be minimal. 


Table A.1. IES levels of evidence for What Works Clearinghouse practice guides 


STRONG MODERATE MINIMAL 


CRITERIA =AYACe(=\ perc t= Ky) =AYACe(=Yalexe Ml =t- [1-1 =AYACe(=Yalexe Wl =t- J) 
Extent of evidence 


The research includes The research includes at least one study 
studies that meet WWC _ that meets WWC standards and provides 
standards and provide a_ a “small” extent of evidence.° 

“medium to large” extent 
of evidence.? 


The research may 
include evidence from 
studies that do not 
meet the criteria for 
moderate or strong 
evidence. 


Effects on relevant 
outcomes 


The research shows, The research shows, for the relevant There may be weak 
for the relevant outcome domain(s), a preponderance of or contradictory 
outcome domain(s), evidence of “positive effects” or “potentially evidence of effects. 
a preponderance of positive effects.” Contradictory evidence 
evidence of “positive of “negative effects” or “potentially 
effects” without negative effects” must be discussed and 
contradictory evidence considered with regard to relevance to the 
of “negative effects” or scope of the guide and the intensity of the 
“potentially negative recommendation as a component of the 
effects.” intervention evaluated. 


WWC 2021007 Designing and Delivering Career Pathways at Community Colleges | Appendix A | 63 


CRITERIA 


STRONG 


MODERATE 
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MINIMAL 


Relevance to 
scope 


Relationship 
between 

research and the 
recommendations 


Panel confidence 


Role of expert 
opinion 


When assessment 
is the focus of the 
recommendation 


=AYACe(=Yalexc Wl = t= 1-1) 


The research has direct 
relevance to the scope— 
relevant context, sample, 
comparisons, and 


outcomes evaluated. 


Evidence Base 


Relevance to scope may vary. At least 
some research is directly relevant to the 
scope. 


=AYACe(=Yalexe Wm =¥- 1-1-1 


The research may be 
out of the scope of the 
practice guide. 


The research includes 
a direct test of the 
recommendation or the 
recommendation is a 
major component of the 
interventions tested in 
the studies. 


Intensity of the recommendation as a 


component of the interventions evaluated 
in the studies may vary. 


Studies for which 

the intensity of the 
recommendation 

as a component of 
the interventions 
evaluated in the 
studies is low, and/or 
the recommendation 
reflects expert opinion 
based on reasonable 
extrapolations from 
research. 


The panel has a high 


degree of confidence 
that a given practice is 
effective. 


The panel determines that the research 
does not rise to the strong level of 
evidence but is more compelling than a 
minimal level of evidence. 


The panel may not be confident about 
whether the research has effectively 
controlled for other explanations or 
whether the practice would be effective in 
most or all contexts. 


In the panel’s opinion, 
the recommendation 
must be addressed 
as part of the practice 
guide; however, the 
panel cannot point to 
a body of research 
that rises to the 
moderate or strong 
level of evidence. 


Not applicable. 


Not applicable. 


Expert opinion 
based on defensible 
interpretations of 
theory. 


generalizability. 


¢ American Educational Research Association, American Psychological Association, & National Council on Measurement in 


Education (1999) 


For assessments, 
research meets 

the standards of 

The Standards for 
Educational and 
Psychological Testing.° 


For assessments, research provides 


evidence of reliability that meets 

The Standards for Educational and 
Psychological Testing but with evidence 
of validity from samples not adequately 
representative of the population on which 
the recommendation is focused. 


Not applicable. 


Note: A recommendation must satisfy all applicable requirements in the same column for the WWC to characterize the practice 
as supported by the evidence base at that level. 


4 This includes randomized controlled trials (RCTs) and quasi-experimental design studies (QEDs) for this practice guide. 


> The research may include studies generally meeting WWC group design standards and supporting the effectiveness of 
a program, practice, or approach with small sample sizes and/or other conditions of implementation or analysis that limit 
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A Final Note About IES Practice Guides 


Expert panels try to build a consensus, forging statements that all its members endorse. Practice 
guides do more than find common ground; they create a list of actionable recommendations. Where 
research clearly shows which practices are effective, the panelists use this evidence to guide their 
recommenda-tions. However, in some cases, research does not provide a clear indication of what 
works. In these cases, the panelists’ interpretation of the existing, but incomplete, evidence plays an 
important role in developing the recommendations. 
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Appendix B: Methods and Processes for Developing This 


Practice Guide 


Phase 1: Selecting the Panel; Establishing a Review Protocol 


Expert Panel. The WWC established a 6-member expert panel to advise on the development of the 
practice guide. The panel consisted of researchers who were at the forefront of career and technical 
education research and practitioners with experience in implementing career pathways interventions. 


Practice Guide Review Protocol. The WWC worked with the expert panel to develop the practice 
guide review protocol, available at https://ies.ed.gov/ncee/wwc/Document/1294, which clarifies the 
practice guide’s purpose and scope. Five questions were identified to guide the literature search and 
the evidence review effort: 


¢ Which CTE interventions are effective at helping students progress toward the completion of an 
industry-recognized credential (e.g., transition from adult basic education [ABE] or apprenticeship 
to postsecondary education)? 


¢ Which CTE interventions are effective at helping students obtain an associate or a baccalaureate 
degree?!5 


¢ Which CTE interventions are effective at helping students obtain an industry-recognized credential, 
a license, or a certificate? Which CTE interventions are effective at helping students achieve 
technical skill proficiency? 


¢ Which CTE interventions are effective at helping students obtain and/or retain employment? 
¢ Which CTE interventions are effective at increasing earnings? 


The time frame for the literature search was approximately 10 years, from January 1999 to May 2019. 
The eligible sample included participants in a postsecondary workforce development intervention in 
the United States. These participants must be 16 years or older and enrolled in a community college.’° 
Eligible study designs included randomized controlled trials (RCTs) and quasi-experimental design 
studies (QEDs). Studies had to include a postsecondary workforce development intervention with a 
primary focus on developing the technical skills and knowledge required for specific jobs or fields of 
work. Only outcomes that fit into one of eleven outcome domains addressing postsecondary student 
educational and labor market outcomes were eligible for inclusion. The eleven domains were: 


1. Credit accumulation 7. Medium-term employment 
2. Postsecondary degree attainment 8. Long-term employment 
3. Attainment of a credential 9. Short-term earnings 


(certification, license, or certificate) 10. Medium-term earnings 


4. Academic performance 11. Long-term earnings 
5. Technical skill proficiency 


6. Short-term employment 


For additional details, the protocol can be accessed on the What Works Clearinghouse website.” 
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Phase 2: Literature Search and Review 


A targeted yet comprehensive search of electronic databases was conducted using keywords focused 
on eligible CTE intervention components, population, setting, study design, and outcomes. Panel 
members also recommended studies that could potentially contribute to the guide. 


A total of more than 16,000 records were identified and screened using a multi-stage screening 
process to determine if they focused on CTE interventions and met the eligibility criteria detailed in 
the protocol (i.e., eligible CTE interventions, population, setting, study design, and outcomes). This 
screening process resulted in 122 eligible studies that were reviewed using WWC 4.0 group design 
standards."* For a study to meet WWC standards, at least one contrast must meet standards with 

or without reservations. See Figure B.1. for the number of records that went through the screening 
and eligibility process and the number of studies that were reviewed with the corresponding WWC 
evidence ratings. 


Figure B.1. Studies identified, screened, and reviewed for this practice guide 


Records identified 
(n=16 | 


Records screened in for Records excluded 
FSeeeing topic relevance (n=357) (n= 16,026) 


Records eligible for review Records ineligible for review 
—_———_—*# 
(n=122) (n=235) 


Studies that meet WWC 


=AV(e(-VaersMae-lilale] Standards without 
reservations (n=14) 


Ko f=Yahdnicer-iaeyal 


Eligibility 


Studies that do not meet 
Studies that meet WWC WWC Standards (n=103) 


Standards with reservations 
(n=7) 


Note: One record (Martinson et al., 2016) includes three studies that meet WWC Standards without reservations. 
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Phase 3: Generating the Recommendations 


WWC staff conducted a detailed examination of the studies that meet WWC standards to identify 
practices that played a role in each intervention. In conjunction with the WWC, the panel identified 
five recommendations that were grounded in evidence provided by the 21 studies that meet WWC 
standards. The panel then suggested ideas for carrying out the recommendations. 


Phase 4: Drafting the Practice Guide 


WWC staff worked with the panel to further expand and clarify each recommendation and delineate 
how to implement each recommendation. WWC staff then used an iterative process to draft the 
recommendations, soliciting feedback from the panel and revising as needed at each stage. WWC staff 
also compiled the level of evidence for each recommendation and drafted the technical appendices. 
The practice guide underwent several rounds of review, including IES external peer review (as 
described in Appendix A). 
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Conducting Reviews of Eligible Studies 


WWC-certified staff reviewed 21 studies to assess the quality of evidence supporting education 
programs and practices, using WWC group design standards version 4.0. The 21 studies that meet 
WWC standards provide the evidence for the recommendations. These studies are bolded in the 
endnotes and in the reference pages. 


Determining Relevance to Recommendations 


Most of the 21 studies provide evidence for more than one recommendation, as the interventions 

in these studies include more than one practice (or component) for improving student outcomes. 

For example, one multi-component intervention might include career pathways (Recommendation 1), 
basic skills instruction (Recommendation 2), comprehensive student supports (Recommendation 4), 
and employment-focused partnerships (Recommendation 5), and thus be used as evidence for all 
four recommendations in this guide. The project staff determined which components were likely to 
cause an effect based on their prominence in the intervention program. Major intervention 
components in each study that meet standards were then assigned to the evidence base for the 
relevant recommendation. In Table C.1, the mapping between each study and the five 
recommendations is presented. 


Table C.1. Mapping between studies and recommendations 


Recommendation 


D ga 
nar fe 
s |$2 joe je# [52 
2 iGo |e [ee oe 
Study SE ln & | im 
Anderson et al. (2017) e e e e 
Bragg et al. (2018) e 
Cook et al. (2018) e e 
Curl et al. (2016) e 
Dunham et al. (2016) e e 
Fein & Beecroft (2006) e e 
Fein & Hamadyk (2018) e e 
Gardiner et al. (2017) e ° e e 
Glosser et al. (2018) e e e 
Hamadyk & Zeidenberg (2018) e e e e e 
Martinson et al. (2018) e e e e e 
Martinson et al. (2016 - Grand Rapids Community College) ° e e 
Martinson et al. (2016 - Kern Community College District) e e e e 
Martinson et al. (2016 - North Central Texas College) e 
Modicamore et al. (2017) e e e e 
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Recommendation 


58 
ois eS fe 
S Ss @p> eZ 58 
= os |2o @ jee 
£ |o5 |xz2 ae |e 
Study S jas ln s a |u & 
Price et al. (2016) e e e 
Price et al. (2017) e 
Price et al. (2018) e e 
Roder & Elliott (2018) e e 
Rolston et al. (2017) e 
Visher & Teres (2011) e 


Determining Relevant Outcomes 


To simplify and focus the synthesis of evidence, the WWC worked with the panel to identify 
which outcome domains were relevant for each recommendation. The relevant domains for each 
recommendation are listed in Table C.2. 


Some of the recommendations are aligned with educational or labor market outcomes 

only. Recommendation 1 is aligned with both educational and labor market outcomes, but 
Recommendations 2, 3, and 4 are aligned with educational outcomes only, and Recommendation 5 is 
aligned with labor market outcomes only. No studies that meet WWC standards included findings in 
either the long-term employment or long-term earnings domains. 


The panel and staff considered only the findings in the predetermined relevant domains when 
determining the level of evidence for each recommendation. For brevity, only findings in relevant 
domains are presented in this appendix. 


Table C.2. Relevant domains for each recommendation 


nc-xexeyanlant=yarerstite) aim Recommendations 2-4 Recommendation 5 


Education outcome domains Education outcome domains Labor market outcome domains 
1. Credit accumulation 1. Credit accumulation 6. Short-term employment 
2. Postsecondary degree attainment 2. Postsecondary degree 7. Medium-term employment 
3. Attainment of a credential uu 8. Long-term employment 

(certification, license, or certificate) 3. peel eid ae credential 9. Short-term earnings 

4. Academic performance nee sai as 10. Medium-term earnings 
5. Technical skill proficiency Ae aeatenie penomuanee 11. Long-term earnings 

Labor market outcome domains 5. Technical skill proficiency 


6. Short-term employment 
7. Medium-term employment 
8. Long-term employment 

9. Short-term earnings 

10. Medium-term earnings 

11. Long-term earnings 
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Estimating Fixed-Effects Meta-Analytic Effect Sizes 


As discussed in Appendix A, the level of evidence determination for each recommendation relied on 
the extent of the evidence from the supporting studies. To synthesize the evidence across studies for 
each recommendation, the WWC calculated a weighted fixed-effects meta-analytic mean effect size 
for each relevant outcome domain in which at least two studies had findings."° This pooled estimate, 
which treats all of the studies contributing to that practice recommendation as a single study, means 
the WWC did not rely on a “vote counting” approach to assess evidence of positive effects on any 
relevant outcome. For domains in which only one study had findings, the study’s domain-level effect 
size was used in the level of evidence determination. To calculate the meta-analytic weight, studies 
were weighted by the inverse of the variance of each study’s effect size. Thus, large-scale studies 
received more weight than small-scale studies. The statistical significance of each effect size for 

each outcome domain was calculated using a z-test. For additional information on this process, see 
Appendix H of the WWC Version 4.1 Procedures Handbook. 


To ensure that the resulting effect sizes were statistically independent, only one contrast from each 
study was included in the analysis.“° In the case of multiple-contrast studies, only the findings from 
the contrast most relevant to the recommendation were included in the meta-analytic effect size 
calculation. Relevant contrasts that compared the effectiveness of two treatments were excluded from 
the meta-analysis. 


For consistency, the meta-analytic effect size calculation for each domain is based on outcomes 
measured closest to the end of the intervention.” The effect sizes for each domain are listed in Tables 
C.4, C.6, C.8, C.10, and C.12. 


Recommendation 1: Intentionally design and structure career pathways 
to enable students to further their education and enter and advance in 
employment. 


Rationale for Level of Evidence: Moderate 


WWC staff and the expert panel assigned Recommendation 1 a moderate level of evidence, based on 

12 studies of interventions with career pathways components. Eight of the studies“? meet WWC group 
design standards without reservations, and the other four studies"* meet WWC group design standards 
with reservations. 


Across the 12 studies, there were findings in five outcome domains (Table C.3) even though all 11 
outcome domains were relevant for this recommendation. Four domains had statistically significant, 
positive meta-analytic effect sizes: credit accumulation (g = 0.03, p < 0.05), industry-recognized 
credential attainment (g = 0.46, p < 0.01), short-term earnings (g = 0.35, p < 0.01), and short-term 
employment (g = 0.19, p < 0.01). One domain had a statistically significant, negative meta-analytic 
effect size: postsecondary degree attainment (g = -1.40, p < 0.01). 
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Table C.3. Domain-level effect sizes across the 12 studies supporting Recommendation 1 


Number Lob Oxo) nile (-aler=) 

Domain of studies Effect size Talk=y ave. | p Value 
Credit accumulation 3 0.03 0.00, 0.05 < 0.05 
Postsecondary degree attainment 2 -1.40 -—1.57, -1.22 < 0.01 
Industry-recognized credential attainment 9 0.46 0.44, 0.49 < 0.01 
Short-term employment 6 0.19 0.14, 0.23 < 0.01 
Short-term earnings 6 0.35 0.29, 0.40 < 0.01 
Note: Significant findings are bolded. ns = nonsignificant findings. All effect sizes were calculated using a fixed effects meta- 
analytic effect size across studies. None of the 12 studies included findings that meet WWC standards in the following outcome 
domains: (1) academic performance; (2) technical skill proficiency; (3) medium-term employment; (4) long-term employment; 
(5) medium-term earnings; or (6) long-term earnings. 


Evidence from all 12 studies provides a direct test of the recommendation, as career pathways are 

a major component of the interventions. In Martinson et al. (2018), the intervention included a 
sequenced pathway of occupational training and remediation of basic skills, followed by sequenced, 
college-level courses. In Hamadyk & Zeidenberg (2018), the intervention adapted and linked existing 
healthcare bridge programs into three 1-semester Patient Care Pathway academies. The intervention 
included a combination of occupational training and remediation of basic skills, following sequenced, 
ready-made courses. In Fein & Hamadyk (2018), the intervention supported participants in five 
pathways in the information technology and financial services sectors. In Martinson et al. (2016; 
Grand Rapids Community College and Kern Community College District), the interventions included 
a sequenced series of connected training courses linked to in-demand occupations and integrated 
occupational skills and basic skills instruction. In Gardiner et al. (2017), 16 existing programs were 
mapped into five pathways. In Price et al. (2016), colleges created new manufacturing pathways and 
modified existing pathways to enable participants to earn short-term credentials that stack toward 
one-year and two-year technical diplomas, and sometimes toward associate’s degrees. In Glosser et al. 
(2018), the intervention offered entry-level coursework and certificates that were part of a structured 
career pathway and intended to lead to additional college-level education or training. In Anderson et 
al. (2017), there were 154 integrated career pathways programs that were credit-bearing and stackable. 
In Modicamore et al. (2017), the intervention offered accelerated, integrated basic skills, occupational 
skills, and job readiness training to prepare job seekers for high-demand occupations and further 
education. In Dunham et al. (2016), the intervention created career pathways in the shale oil and 

gas industry by expanding existing training options and developing stackable credential training 
programs. In Price et al. (2018), the colleges developed and enhanced healthcare career pathways. 


The collection of studies demonstrate high external and moderate internal validity and a 
preponderance of positive impacts. In all of the studies supporting this recommendation, career 
pathways are a major component of the tested intervention. Therefore, the expert panel has assigned 
a moderate level of evidence to this recommendation. This rating is supported by the strength of the 
evidence according to the following criteria: 


¢ Internal and External Validity of Supporting Evidence. The average effect sizes for four of the 
five outcome domains (industry-recognized credential, credit accumulation, short-term earnings, 
and short-term employment) are positive and statistically significant, and each outcome domain 
average is based on more than one study with a total sample size above 350. However, the average 
effect sizes for the positive outcome domains are predominantly derived from studies that meet 
WWC group design standards with reservations. 
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- Consistency of Effects on Relevant Outcomes. The average effect size is statistically significant 
and negative for one of the five outcome domains (postsecondary degree attainment), but it is 
statistically significant and positive for the other four outcome domains (industry-recognized 
credential, postsecondary degree attainment, credit accumulation, short-term earnings, and 
short-term employment). 


¢ Relationship Between the Evidence and Recommendation. In all 12 of the studies used to 
support Recommendation 1, career pathways are a major component of the tested intervention. 


Table C.4. Studies providing evidence for Recommendation 1: Intentionally design and 
structure career pathways to enable students to further their education, secure a job, and 
advance in employment 


Outcome Domain 
and WWC 


Comparison Calculated Effect 
Participants? | Setting TalcVarccland(olamexeyacelid(ola Mmmm ereyaleliaeya Size 


Anderson et al. (2017) 42,984 Community Accelerating Opportunity Business-as- Credit 
students colleges (AQ): Community and usual: The accumulation: 


Additional source: 


Anderson et al. (2015) 9 @verage age- 
28.3 years 


and adult technical colleges 
education that were in the AO 
settings in intervention condition 

4 states (IL, developed or modified 
KS, KY, LA) | existing programs that 
offered career pathways 
for in-demand jobs. 

A major component 

of AO was integrated 
instruction, where 

both basic skills and 
CTE instructors taught 
the same class with 

at least 25 percent 
overlap. Students also 
received additional 
services, including 
advising, navigational, 
and financial supports, 
to help them progress 
through the program. 


comparison g = -0.03 
group students 
were drawn Industry- 
from the same _ "ecognized 
recruitment credential 
sources (adult attainment: 
education, g = 0.49* 
eevee Short-term 
education, or ieee 
earnings: 


CTE) as the one ed 
: P Illinois participants 
intervention 
earned $125 
group, but they 
: more on average 
did not have the . 
: in a quarter than 
opportunity to . 
a business-as-usual 
participate ; i 
: did. Louisiana 
in AO. i 
participants 
earned $194 
more on average 
in a quarter than 
business-as-usual 


QED 


Meets WWC group 
design standards with 
reservations 


did 
One relevant feature Shorten 
of this program was ; 
the use of an advisory Spay nent 
g = 0.09* 


board for career pathway 
development. 
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Outcome Domain 
and WWC 
Calculated Effect 
Size? 


Dunham et al. (2016) 
QED 


Meets WWC group 
design standards with 
reservations 


Participants? 
353 students 


average age: 

35.2 years 

° 42.5% are 
older than 
25 years 


Setting 


4 technical 
colleges in 3 intervention is a 


states (PA, 
OH, TX) 


ShaleNET: The 


community-based 

job training program 

that offered stackable 
credential training 
programs in the oil and 
gas industry. Post- 
training internships 

and other work-based 
learning were sometimes 
offered. Support services 
included job search/ 
placement services and 
career counseling. 


One relevant feature of 
this program was the 
emphasis on identifying 
and assessing industry 
needs when designing 
the career pathways. 


Workforce 
assistance: 
The comparison 
group was 
drawn from the 
population of 
individuals from 
the surrounding 
counties who 
received 
assistance 

with finding 
employment 
from federally 
funded 
workforce 
programs in 
Pennsylvania 
during the same 
time period. It 
excluded all 
individuals who 
received any 
training. 


Short-term 
earnings: 
g = 6.46* 


Short-term 
employment: 
g = 0.38* 
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Study 


Fein & Hamadyk 
(2018) 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 

2,544 young 

adults (ages 

16-24) 

° 57.2% are 
21 to 24 

° 42.8% are 
18 to 20 


Setting 

8 local 
offices that 
partner with 
community 


NY, RI, WA) 


TaiteyaV@-vald(olamexeyatelid(eya) 
Year Up: The 
intervention provided 
six months of tuition- 
free full-time training 

in IT and financial 
services followed 

by six months of an 
internship. Young adults 
participated in a cohort- 
learning experience 
that integrated 
occupational, general, 
and contextualized 
basic skills. Support 
services included one- 
on-one advising, tutoring, 
community mentorship, 
case management, 
counseling, financial 
supports, and 
employment support 
services. 


One relevant feature 

of this program 

was consulting and 
collaborating with 
employers and industry 
partners to review 
relevant labor market 
information as part 

of career pathway 
development. The 
program was also 
designed with an intake 
process assessing 
career aptitude and 
goals. 


Comparison 
foreyareliiteyal 
Business-as- 
usual: 
Comparison 
group members 
could receive 
other training, 
education, and 
supports in the 
community. 
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Outcome Domain 
and WWC 
Calculated Effect 
Size? 

Short-term 
earnings: 

g = 1.00* 
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Gardiner et al. (2017) 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
1,217 
students 


79% are 25 

years or older 

° 8.4% are 
20 or 
younger 

* 13.0% are 
21 to 24 

° 31.7% are 
25 to 34 

* 46.9% are 
35 or older 


Setting 

1 community 
college 

and local 
workforce 
agency in 
Arizona 


Talc=yavcclalice)amexeyareliareyal 


Pathways to Healthcare: 
The intervention 


offered occupational 
training programs in the 
healthcare industry. A 10 
week basic skills “bridge 
class” was offered 

to participants who 
needed to increase their 
skills prior to starting a 
training program. The 
intervention also offered 
advising; scholarships 
for tuition, books, and 
program supplies; job 
assistance services; and 
peer support groups. 


Relevant features of this 
program included clearly 
defined entry and exit 
points, a simplified and 
more accessible set of 
career pathways, anda 
comprehensive, holistic 
intake process. 


Comparison 
Condition 
Business-as- 
usual: The 
comparison 
group had 
access to 
services 
including 
standard 
assessment, 
basis skills 
education, 
occupational 
training 
programs, and 
support services 
(e.g., advising 
and tutoring), 
and employment 
services that 

are regularly 
available at 

the community 
college. 


Appendix C 


Outcome Domain 
and WWC 


Calculated Effect 
Size? 
Credit 
accumulation: 
g = -0.07 


Industry- 
recognized 
credential 
attainment: 
g = 0.20* 


Short-term 
earnings: 
g = -0.03 
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(oye T Kero) pat-m Bloyait-lin) 


and WWC 
Comparison Calculated Effect 
Participants? | Setting TalcVarccland(olamexeyacelid(ola Mmmm ereyaleliaveya Size” 
Glosser et al. (2018) 631 students 3 community |-BEST: The I-BEST Business-as- Industry- 
colleges in _ intervention includes usual: recognized 
RCT 63% are 25 Washington courses that are part of | Comparison credential 
Meets WWC group years a older a structured pathway, group members atainment 
design standards ° 22.2% team-teaching between could not access g = 0.49 
: : are 20 or basic skills instructors the I-BEST 
without reservations ; ; : 
younger and occupational intervention and 
° 14.9% are instructors, and courses at the 
21 to 24 reimbursement to the three colleges; 
° 29.8% are college for 1.75 FTE per however, they 
25 to 34 student. This intervention could access 
¢ 33.2% are also included dedicated other education 
35 or older academic advisors/case and training 


managers, career/college opportunities 
navigation services, and available to 
financial assistance for — them, including 
training-related costs and non-l-BEST 
transportation. courses and 
the I-BEST 
intervention at 
other colleges. 
Both treatment 
and comparison 
group members 
could access 
general college 
advising, 
tutoring, 
financial aid 
services and 
job assistance 
services that 
were available 
to all students at 
the colleges. 
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Outcome Domain 


and WWC 
Comparison Calculated Effect 
Participants? | Setting TalcVarcclahd(olamexeyacelid(ola mmm ereyaleliaeya Size 
Hamadyk & 943 students 1 community WTA Connect: Business- Industry- 
Zeidenberg (2018) college in Students enrolled in as-usual: recognized 
69% are 25 jowa tuition-free remedial Students inthe credential 
RCT ao ee older and occupational comparison attainment: 
yes s0 certificate courses in group had g = 0.25* 
yee ae es ee are 20 or healthcare, advanced access to 
ie : younger manufacturing, and standard Short-term 
SO es oro * 16.4% are administrative support. | courses and earnings: 
21 to 24 Math and reading basic student support 9 = 0.03 
* 27.7% are skills training was also _ services, 
25 to 34 offered. Support services including 
* 41.6% are included “achievement — standard 
35 or older coaches” or advising basic skills 
support, transportation remediation 
assistance, screening software and 
for public benefits, life retest system to 
skill workshops, and reapply to Des 
employment support Moines Area 
services. Community 
College. 
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Outcome Domain 


and WWC 
Comparison Calculated Effect 
Study Participants? | Setting TaicVarcclahd(olamexeyacelid(ola Mmm ereyaleliaeya Size” 
Martinson et al. (2018) 800 students Non- Carreras en Salud: Business-as- Industry- 
profit and Students enrolled in one — usual: recognized 
RCT 56% are 25 community _ or more one-semester Students in the — credential 
years or older college healthcare academies comparison attainment: 
Mest VIVE qioue ° 17.9% rtnership — that provided them with roup had = 0.60* 
design standards are 20 or en Ne aa pln ca 
WHROHP Te seranOne in Illinois academic preparation, access to 
younger an accelerated path standard Short-term 
* 26.6% are through remediation in courses and earnings: 
21 to 24 occupational courses, services at g = -0.04 
* 34.1% are and integrated basic Instituto or other 
25 to 34 skills and remediation local community 
* 21.4% are courses. Support colleges or 
35 or older services included one- _ institutions, 


on-one advising, financial including basic 
support, childcare skills and 

and transportation occupational 
assistance, and training classes, 
employment support and standard 
services. financial aid 


assistance, 
One relevant feature employment, 


of this program was and advising 
the identification and services. 
assessment of industry 

needs when designing 

the career pathways, 

using an advisory 

board to inform career 

pathway development, 

designing pathways 

with clearly defined 

entry and exit points, 

and carefully assessing 

the scope and time 

needed for curriculum 
redevelopment. 
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Martinson et al. (2016) 


Grand Rapids 
Community College 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 


274 students 


average age: 

40.4 years 

° 5.4% are 
21 or 
younger 

° 14.1% are 
22 to 29 

* 28.5% are 
30 to 39 

° 52.0% are 
40 or older 


Setting 

1 community 
college in 
Michigan 


Talc=yavccvalice)amexeyareliareyal 


GRCC Pathways to 
Prosperity: All Green 
Jobs-Health Care 
interventions included 

a sequenced series 

of connected training 
courses linked to in- 
demand occupations and 
integrated occupational 
skills and basic skills 
instruction. Supportive 
services included 
financial assistance, 
tutoring, advising, and 
employment support 
services. At Grand 
Rapids Community 
College, 8-week training 
courses were offered in 
green-related sectors. 
Basic skills training was 
provided to those who 
needed it before starting 
training. 


Relevant features of this 
program included the 
use of an advisory board 
during career pathway 
development and a 
holistic intake process. 


Comparison 
foreyateliiteyal 
Business-as- 
usual: Across 
community 
college sites, 
comparison 
group members 
could not 
access the 
intervention’s 
grant-funded 
services, but 
could access 
other similar 
services 
available in the 
community. 


Appendix C 


Outcome Domain 
Fe lavem Ayes 


Calculated Effect 
Size? 
Postsecondary 
degree attainment: 
g = 0.45* 


Industry- 
recognized 
credential 
attainment: 
g = 1.00* 


Short-term 
employment: 
g=0.18 
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Outcome Domain 


Flare mA (03 
Comparison Calculated Effect 
Study Participants? | Setting TaicVarccvahd(olamexeyacelid(ola Mmm ereyaleliareya Size 
Martinson et al. (2016) 816 students 1 community KCCD Clean Energy Business-as- Postsecondary 
college in Center: All Green usual: Across degree attainment: 
Kern Community average age: California Jobs-Health Care community g = -2.24* 
College District 32.3 years interventions included _ college sites, 
RCT * 19.2% a sequenced series comparison Industry- 
are 21 or of connected training group members recognized 
Meets WWC group oo courses linked to in- could not credential 
design standards ° 30.8% are demand occupations and access the altelnment 
“nA PES ARISIITS 22 to 29 integrated occupational _intervention’s 9 = 1.39 
* 24.8% are skills and basic skills grant-funded 
30 to 39 instruction. Supportive _ services, but Shounen ; 
* 25.2% are services included could access employment: 
40 or older financial assistance, other similar ee 


tutoring, advising, and services 
employment support available in the 
services. At Kern community. 
Community College 

District, all participants 

started with a 6-week 

foundational training that 

integrated occupational 

training, basic skills, and 

employment support 

services. 


Relevant features of this 
program included the 
use of an advisory board 
during career pathway 
development, a core 
curriculum that applies 
across multiple sectors, 
and a holistic intake 
process. 
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(oye Kexe) pat-m Bloyaar-lin) 
Flare m Alles 


Comparison Calculated Effect 
Participants? | Setting TaicVarceland(olamexeyacelid (ola Mmm ereyaleliarey a Size 


2,168 9 sites Accelerating Connections Business-as- Industry- 


Modicamore et al. 


(2017) students across 4 to Employment (ACE): usual: The recognized 
different The ACE provides comparison credential 
RCT average age: states (CT, training that integrates condition attainment: 
WRG aa 35.5 years GA, MD, TX) basic and vocational included access g = 0.45* 
é oer * 78% are skills instruction. It also to any non-ACE 
gescn SUSIE older than offers support services services atthe | Short-term 
rout reso valions 25 years such as individual local workforce &Mployment: 


coaching, tutoring, agency. g = 0.27* 


financial assistance, and 


internship/job readiness enonem 
and placement services earnings: 
, g = 0.02 


Relevant features of 
this program included 
use of an advisory 
board during career 
pathway development, 
ongoing assessment 
to identify additional in- 
demand training, and a 
holistic intake process, 
emphasizing career 
aptitude and goals. 
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Outcome Domain 
and WWC 


Comparison Calculated Effect 
Participants? | Setting TaicVarccland(olamexeacelid (ola Mmmm ereyaleliaveya Size 


3,376 14 ACT for Healthcare: Matched Short-term 


Price et al. (2018) 


students community Students enrolled in business-as- employment: 
QED colleges in a TAACCT funded- usual: g = 0.20* 
SIVA era average age: Wisconsin healthcare program and Students in the 
‘ HOE : 28.5 received at least one- comparison 
eet CUUNGEEME, with + 49% are 25 grant funded support group were 
peselvaboys or younger service. The support matched to the 


* 51% are services offered at each _ intervention 
el or college varied, but could = group and were 
older 


enrolled in other 
healthcare 
programs at 

the consortium 
colleges that 
were not 
supported by the 
TAACCT grant 
during the grant 
period. Both 

the intervention 
and comparison 
groups could 
access other 
supportive 
services not 
funded through 
the grant. 


include embedded in- 
class academic supports, 
out of class academic 
supports such as one 

to one tutoring, and 
non-academic supports 
such as life skills 
training, counseling, and 
employment support 
services. 


Relevant features 
included identifying and 
assessing industry needs 
as part of career pathway 
development and 
designing short-term, 
targeted credentials 
within parent programs. 
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Outcome Domain 
and WWC 


Comparison Calculated Effect 
Participants? | Setting Talc-VarccVahd(olamexeyacelid(ola Mmmm ereyaleliaveya Size 


Price et al. (2016) 6,300 16 Making the Future: Business-as- Credit 
students community — The intervention was usual: The accumulation: 

QED colleges in meant to facilitate comparison g = 0.19* 
average age: Wisconsin — students’ ability to earn = group students 

eae dienes ith ae some s , short-term credentials, were enrolled in Industry- 4 

OL AI or treaimen which typically takes less similar programs "ecognize 
group than 1 year, and “stack” of study offered credential 


by the technical attainment: 
colleges and g = 0.35" 
had access to 
typical courses, 
programs, and 
student support 
services. 


toward 1-year and 2-year 
technical diplomas. The 
intervention included 
stacked and latticed 
manufacturing programs 
in welding, machine 

tool and computer 
numerical control 

(for 3D printing), and 
industrial maintenance. 
Support services varied 
depending on the college 
but included advising, 
counseling, and tutoring. 


Relevant features 
included identifying and 
assessing industry needs 
and using an advisory 
board as part of career 
pathway development. 


Note: Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention ona 
distinct sample. 


@ Sample size represents the maximum number of participants in the study. In some studies, the number of participants varied 
across the outcome measures. 


> Effect sizes presented are from the outcome closest to the end of the intervention. For brevity, only the domain average effect 
size and statistical significance are reported in this table. 


° There was insufficient information reported in Anderson et al. (2017) to generate effect sizes in the short-term earnings 
domain. In the table, coefficients from the analysis models are presented. The authors cited short-term earnings as statistically 
significant. 


* = statistically significant at the .05 level. 
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Recommendation 2: Deliver contextualized or integrated basic skills 
instruction to accelerate students’ entry into and successful completion 
of career pathways. 


Rationale for Level of Evidence: Moderate 


WWC staff and the expert panel assigned Recommendation 2 a moderate level of evidence, based on 
nine studies of interventions that include contextualized or integrated basic skills instruction. Seven 
of the studies'* meet WWC group design standards without reservations, and the other two studies"° 
meet WWC group design standards with reservations. 


Across the nine studies, there were findings in three outcome domains (Table C.5) even though all five 
educational outcome domains were relevant for this recommendation. Two domains had statistically 
significant, positive meta-analytic effect sizes: credit accumulation (g = 0.03, p < 0.05) and industry- 
recognized credential attainment (g = 0.45, p < 0.01). One domain had a statistically significant, 
negative meta-analytic effect size: postsecondary degree attainment (g = -2.24, p < 0.01). 


Table C.5. Domain-level effect sizes across the nine studies supporting Recommendation 2 


Number bom Oxo) ar i(e (= aler=) 
Domain of studies Effect size Talk=yave-1| p Value 
Credit accumulation 4 0.03 0.00, 0.05 | < 0.05 
Postsecondary degree attainment 1 -2.24 —2.45, -2.03 | < 0.01 
Industry-recognized credential attainment 9 0.45, 0.43, 0.47, < 0.01 


Note: Significant findings are bolded. ns = nonsignificant findings. All effect sizes were calculated using a fixed effects meta- 
analytic effect size across studies, except for the postsecondary degree attainment domain. This domain had findings from just 
one study; the effect size and p value presented here are the domain-level average effect size and p value for the individual 
relevant study. None of the nine studies included findings that meet WWC standards in the following outcome domains: (1) 
academic performance; or (2) technical skill proficiency. 


Evidence from eight of the nine studies provides an indirect test of the recommendation, as 
contextualized or integrated basic skills instruction is a major or secondary component of the 
interventions. In Martinson et al. (2018) and Hamadyk & Zeidenberg (2018), basic skills instruction was 
contextualized with occupational content. In Martinson et al. (2016, Kern Community College District), 
the training program provided instruction on workplace safety as well as basic math and electrical 
skills. In Cook et al. (2018), accelerated and contextualized basic skills instruction was provided that 
shortened the number of remediation courses to a single semester. In Price et al. (2016), some sites 
implemented adult education and basic skills courses to connect students to manufacturing programs. 
Some took additional measures and integrated coursework with basic skills instruction. In Glosser et 
al. (2018), the central feature of the intervention was integrated basic skills and vocational education. 
In Anderson et al. (2017), contextualized basic skills and vocational instruction were integrated and 
provided as team teaching. In Modicamore et al. (2017), contextualized learning was the foundation of 
the intervention, and as such the intervention co-taught basic skills and vocational training. 


In one study, Gardiner et al. (2017), the contextualization of basic skills was not a big component of the 
intervention, which was more focused on further training and coordinated supports. 
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The collection of studies demonstrate high external and moderate internal validity and a 
preponderance of positive impacts. In eight of the nine studies supporting this recommendation, 
contextualized or integrated basic skills instruction is a major or secondary component of the 

tested intervention. Therefore, the expert panel has assigned a moderate level of evidence to this 
recommendation. This rating is supported by the strength of the evidence according to the following 
criteria: 


¢ Internal and External Validity of Supporting Evidence. The average effect sizes for two of the 
three educational outcome domains (credit accumulation and industry-recognized credential) are 
positive and statistically significant, and each outcome domain average is based on more than one 
study with a total sample size above 350. However, the average effect sizes for the positive outcome 
domains are predominantly derived from studies that meet WWC group design standards with 
reservations. 


¢ Consistency of Effects on Relevant Outcomes. The average effect size is statistically significant 
and negative for one of the three educational outcome domains (postsecondary degree attainment), 
but it is statistically significant and positive for the other two educational outcome domains (credit 
accumulation and industry-recognized credential). 


¢ Relationship Between the Evidence and Recommendation. In eight of the nine studies used 
to support Recommendation 2, contextualized or integrated basic skills instruction is a major or 
secondary component of the tested intervention. 
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Table C.6. Studies providing evidence for Recommendation 2: Deliver contextualized or integrated basic 
skills instruction to accelerate students’ entry into and successful completion of career pathways 


Outcome Domain 


and WWC 
Comparison Calculated Effect 
Participants? | Setting Talc-Varcclani(oyamexeyacelid(olaMmmereyatelia(eya Size’ 
Anderson et al. (2017) 42,984 Community Accelerating Opportunity Business-as- Credit 
students colleges (AQ): Community and usual: The accumulation: 
Additional source: and adult technical colleges comparison g = -0.03 
Anderson et al. (2015) average age: education __ that were in the AO group students 
QED 28.3 years settings in —_ intervention condition were drawn Industry- 
4 states (IL, developed or modified from the same recognized 
Meets WWC group KS, KY, LA) existing programs that —_— recruitment credential 
design standards with offered career pathways sources (adult attainment. 
FESS for in-demand jobs. education, g = 0.49 
A major component developmental 
of AO was integrated education, or 
instruction, where CTE) as the 
both basic skills and intervention 
CTE instructors taught group, but they 
the same class with did not have 
at least 25 percent the opportunity 
overlap. Students also to participate in 
received additional AO. 


services, including 
advising, navigational, 
and financial supports, 
to help them progress 
through the program. 


Relevant features of 

this study included 

the different team 
teaching models used 

by basic skills and CTE 
instructors, as well as the 
use of braided funding 

to cover the costs of the 
program. 
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Appendix C 


Outcome Domain 
and WWC 
Calculated Effect 


Cook et al. (2018) 
RCT 


Meets WWC group 
design standards 
without reservations 


Gardiner et al. (2017) 
RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
500 students 


55 % are 25 
years or older 
* 23.4% 

are 20 or 
younger 
21.2% are 
21 to 24 
30.0% are 
25 to 34 
25.4% are 
35 or older 


1,217 
students 


79% are 25 

years or older 

* 8.4% are 
20 or 
younger 

* 13.0% are 
21 to 24 

° 31.7% are 
25 to 34 

° 46.9% are 
35 or older 


Setting 

1 community 
college in 
Wisconsin 


1 community 
college 

and local 
workforce 
agency in 
Arizona 


Talc=yavcclalice)amexeyareliareyal 


MATC Patient Care 
Pathway: The Patient 
Care Pathway 
intervention adapted 
and linked existing 
healthcare bridge 
programs into three one- 
semester Patient Care 
Pathway academies that 
combined occupational 
training and basic skills 
remediation. Students 
were also provided with 
dedicated advisors. 


A relevant feature 

of the program was 
the provision of extra 
instructional support in 
response to emergent 
student needs. 


Pathways to Healthcare: 
The intervention 


offered occupation 
training programs in the 
Healthcare Industry. 

A 10-week basic skills 
“bridge class” was 
offered to participants 
who needed to increase 
their skills prior to starting 
a training program. The 
intervention also offered 
advising, scholarships 
for tuition, books, and 
program supplies, job 
assistance services, and 
peer support groups. 


One relevant feature 
of the program was 
the instructor-led study 
groups to provide 
additional student 
support. 


foreyateliateyal 
Business-as- 
usual: The 
comparison 
group 
participants 

had access to 
usual services, 
including 
regular courses, 
advising 
services, and 
other student 
support services. 


Business-as- 
usual: The 
comparison 
group had 
access to 
services 
including 
standard 
assessment, 
basis skills 
education, 
occupational 
training 
programs, and 
support services 
(e.g., advising 
and tutoring), 
and employment 
services that 
are regularly 
available at 
the community 
college. 


Size” 

Credit 
accumulation: 
g=0.10 


Industry- 
recognized 
credential 
attainment: 
g = -0.08 


Credit 
accumulation: 
g = -0.07 


Industry- 
recognized 
credential 
attainment: 
g = 0.20* 
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Outcome Domain 


and WWC 
Comparison Calculated Effect 
Participants? | Setting TalcVarccVahd(olamexeyacelid(ola Mmm ereyaleliaey a Size’ 
Glosser et al. (2018) 631 students 3 community |-BEST: The I-BEST Business-as- Industry- 
colleges in _ intervention includes usual: recognized 
RCT 63% are - Washington courses that are partof © Comparison credential 
years or olaer a structured pathway, group members _ attainment: 
rae Re ae * 22.2% team-teaching between could not access g = 0.49* 
Be eae ee are 20 or basic skills instructors the I-BEST 
younger and occupational intervention and 
* 14.9% are instructors, and courses at the 
21 to 24 reimbursement to the three colleges; 
* 29.8% are college for 1.75 FTE per however, they 
25 to 34 student. This intervention could access 
* 33.2% are also included dedicated other education 
35 or older academic advisors/case and training 
managers, career/college opportunities 
navigation services, and available to 
financial assistance for — them, including 
training-related costs and non-l-BEST 
transportation. courses and 
the I-BEST 
Relevant features of intervention at 
this program included other colleges. 
the use of different Both treatment 
models of team and comparison 
teaching, monitoring and group members 
assessment strategies could access 
for the implementation general college 
of these, as well as extra advising, 
supports for basic skills tutoring, financial 
instruction. aid services and 
job assistance 
services that 
were available 
to all students at 
the colleges. 
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Outcome Domain 


and WWC 
Comparison Calculated Effect 
Participants? | Setting TalcVarcclahd(olamexeacelid(ola Mmm ereyaleliaeya Size 
Hamadyk & 943 students 1 community WTA Connect: Business-as- Industry- 
Zeidenberg (2018) college in Students enrolled in usual: recognized 
69% are 25 _—s lowa tuition-free remedial Students inthe credential 
RCT years or older and occupational comparison attainment: 
Meets WWC group * 14.3% certificate courses in group had g = 0.25* 
siesta Seaneee are 20 or healthcare, advanced access to 
WHEHOUE eee niehone younger manufacturing, and standard 
* 16.4% are administrative support. courses and 
21 to 24 Math and reading basic — student support 
° 27.7% are skills training was also services, 
25 to 34 offered. Support services including 
° 41.6% are included “achievement — standard 
35 or older coaches” or advising basic skills 
support, transportation remediation 
assistance, screening software and 
for public benefits, life retest system to 
skills workshops, and reapply to Des 
employment support Moines Area 
services. Community 
College. 
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Comparison 
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Outcome Domain 
and WWC 
Calculated Effect 


Participants? | Setting TaicVarccland(olamexeyacelid (ola Mmm ereyaleliareya Size’ 
Martinson et al. (2018) 800 students Non- Carreras en Salud: Business-as- Industry- 
profit and Students enrolled in one usual: recognized 
RCT 56% are 25 community _ or more one-semester Students inthe credential 
years or older college healthcare academies comparison attainment: 
Heh has ° 17.9% rtnership — that provided them with roup had = 0.60* 
design standards are 20 or en Ne aa pln aaa 
Without reservalons in Illinois academic preparation, access to 
younger an accelerated path standard 
* 26.6% are through remediation in courses and 
21 to 24 occupational courses, services at 
* 34.1% are and integrated basic Instituto or other 
25 to 34 skills and remediation local community 
* 21.4% are courses. Support colleges or 
35 or older services included one- _ institutions, 
on-one advising, financial including 
support, childcare basic skill and 
and transportation occupational 
assistance, and training classes, 
employment support and standard 
services. financial aid 
assistance 
One relevant feature of |= employment, 
this program was the use_ and advising 
of a bridge to accelerate _ services. 
students’ progression 
through remediation 
courses. Another 
relevant feature was 
the use of curriculum 
specialists to identify 
relevant topics and to 
provide instructor support 
for contextualized 
instruction. 
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Comparison 
oes aTelittoya 


Appendix C 


Outcome Domain 
and WWC 
Calculated Effect 
Size 


Participants? 


Setting 


Martinson et al. (2016) 816 students 1 community KCCD Clean Energy Business-as- Postsecondary 
college in Center: All GJ-HC usual: Across degree attainment: 
Kern Community average age: California interventions included community g = -2.24* 
College District 32.3 years a sequenced series college sites, 
RCT * 19.2% of connected training comparison Industry- 
are 21 or courses linked to in- group members recognized 
Meets WWC group ; Son ae demand occupations and could not access credential 
design standards Soin 3 : integrated occupational the intervention’s atalnnent 
without reservations skills and basic skills grant funded g = 1.39 
* 24.8% are instruction. Supportive services, but 
30 to 39 services included could access 
* 25.2% are financial assistance, other similar 
40 or older tutoring, advising, and services 
employment support available in the 
services. At KCCD, all community. 
participants started with 
a six-week foundational 
training that integrated 
occupational training, 
basic skills, and 
employment support 
services. 
Modicamore et al. 2,168 9 sites Accelerating Connections Business-as- Industry- 
(2017) students across 4 to Employment (ACE): usual: The recognized 
different The ACE provides comparison credential 
RCT average age: states (CT, training that integrates condition was attainment: 
35.5 years GA, MD, TX) basic and vocational business as g = 0.45* 
Megs WG aioe * 78% are kills instruction. It also usual which 
design standards older than ff ct he ladad 
WHHGEreSene none a offers support services _ included access 
years such as individual to any non-ACE 
coaching, tutoring, services at the 
financial assistance, and _ local workforce 
internship/job readiness agency. 
and placement services. 
One relevant feature of 
this program was the 
use of team teaching 
to identify student 
challenges faster. 
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Outcome Domain 
and WWC 
Calculated Effect 
Size 


Comparison 
oxeyaTelittoya 


Price et al. (2016) 6,300 16 Making the Future: Business-as- Credit 
students community — The intervention was usual: The accumulation: 
QED colleges in meant to facilitate comparison g = 0.19* 
average age: Wisconsin — students’ ability to earn group students 
Meets WE group ; 26.6 years short-term credentials, | were enrolled in — Industry- 
SS aE) for treatment which typically can be similar programs recognized 
reservations ; . 
group earned in less than one _ of study offered credential 
year, and “stack” toward by the technical attainment: 
one-year and two-year colleges and g = 0.35* 


technical diplomas. The 
intervention included 
stacked and latticed 
manufacturing programs 
in welding, machine tool 


had access to 
typical courses, 
programs, and 
student support 
services. 


and computer numerical 
control (CNC - i.e., 

for 3D printing), and 
industrial maintenance. 
Support services varied 
depending on the college 
but included advising, 
counseling, and tutoring. 


Relevant features of this 
program included the use 
of intake assessments 

to inform the types of 
instruction needed, 

the use of bridges to 
accelerate student 
progression through 
remedial coursework, 
and the provision of extra 
supports to supplement 
basic skills instruction. 


Note: Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention on a 
distinct sample. 


@ Sample size represents the maximum number of participants in the study. In some studies, the number of participants varied 
across the outcome measures. 


> Effect sizes presented are from the outcome closest to the end of the intervention. For brevity, only the domain average effect 
size and statistical significance are reported in this table. 


* = statistically significant at the .05 level. 
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Recommendation 3: Offer flexible instructional delivery schedules and 
models to improve credit accumulation and completion of non-degree 
credentials along career pathways. 


Rationale for Level of Evidence: Minimal 


WWC staff and the expert panel assigned Recommendation 3 a minimal level of evidence, based on six 
studies of interventions that include flexible instructional delivery models. Five of the studies“* meet 
WWC group design standards without reservations, and the other study” meets WWC group design 
standards with reservations. An additional four studies“* include flexible instructional delivery models 
but do not report any educational outcomes. 


Across the six studies, there were findings in all five educational outcome domains (Table C.7). 

Two domains had statistically significant, positive meta-analytic effect sizes: postsecondary degree 
attainment (g = 0.32, p < 0.01) and industry-recognized credential attainment (g = 0.33, p < 0.01). The 
other domains (credit accumulation, academic performance, and technical skill proficiency) did not 
have statistically significant meta-analytic effect sizes. 


Table C.7. Domain-level effect sizes across the six studies supporting Recommendation 3 


Number bob Oxo) ar i(e (= aler=) 
Domain of studies Effect size alk=y ave: | p Value 
Credit accumulation 2 -0.04 —0.12, 0.05 | ns 
Postsecondary degree attainment 2 0.32 0.23, 0.41 | < 0.01 
Industry-recognized credential attainment 3 0.33 0.24, 0.41 | < 0.01 
Academic performance 1 0.06 ~0.07, 0.19 ns 
Technical skill proficiency 1 0.36 —0.04, 0.75 ns 
Note: Significant findings are bolded. ns = nonsignificant findings. All effect sizes were calculated using a fixed effects meta- 
analytic effect size across studies, except for the academic performance and technical skill proficiency domains. These domains 
had findings from just one study; the effect sizes and p values presented here are the domain-level average effect sizes and p 
values for the individual relevant study. Across the six studies, there were findings in all five educational outcome domains. 


Evidence from all six studies provides an indirect test of the recommendation, as flexible instructional 
delivery models are a secondary component of the interventions. In Martinson et al. (2018), the 
structure and content of the basic skills instruction accelerated remediation, flexible scheduling was 
offered, and tutoring was offered. In Hamadyk & Zeidenberg (2018), basic skills remediation courses 
were offered as self-paced, online courses that could be taken in an instructor-supervised lab or on 
their own. In Fein & Beecroft (2006), the intervention offered flexible scheduling, small class sizes, 
and self-paced courses. In Visher & Teres (2011), active and project-based learning strategies were used 
alongside flexible scheduling. In Curl et al. (2016), the intervention focused on high-fidelity simulations 
to supplement clinical experiences, which offered flexibility and efficiency in students’ learning 
progression. In Gardiner et al. (2017), students chose the format of their basic skills program, either 
traditional format or a self-paced course, depending on their timeline and priorities. 


The collection of studies demonstrate high external validity, low internal validity, and consistent 
positive impacts. In all of the studies supporting this recommendation, flexible instructional delivery 
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models are a secondary component of the tested intervention. Therefore, the expert panel has 
assigned a minimal level of evidence to this recommendation. This rating is supported by the strength 
of the evidence according to the following criteria: 


¢ Internal and External Validity of Supporting Evidence. The average effect sizes for two of the 
five educational outcome domains (postsecondary degree attainment and industry-recognized 
credential) are positive and statistically significant, but the average effect sizes for the other three 
educational outcome domains (credit accumulation, academic performance, and technical skill 
proficiency) are indeterminate. Three of the five educational outcome domain averages are based 
on more than one study with a total sample size above 350. 


¢ Consistency of Effects on Relevant Outcomes. The average effect size is not statistically 
significant and negative for any of the five educational outcome domains (credit accumulation, 
postsecondary degree, industry-recognized credential, academic performance, or technical 


skill proficiency). 


¢ Relationship Between the Evidence and Recommendation. In all six of the studies used to 
support Recommendation 3, flexible instructional delivery models are a secondary component of 
the tested intervention. 


Table C.8. Studies providing evidence for Recommendation 3: Offer flexible instructional delivery 
schedules and models to improve credit accumulation and completion of non-degree credentials along 


career pathways 


Comparison 


Outcome Domain 
and WWC 
Calculated Effect 


Curl et al. (2016) 
QED 


Meets WWC group 
design standards with 
reservations 


Participants? 
124 students 


age is not 
reported in the 
study 


Setting 


2 community 
colleges 
and 1 public 
university in 
Texas 


TalccYavc-aldle)amexeyareliareya) 


Nursing STRIPES: 
The intervention 


offered 20 four-hour 
simulation modules 
integrated into one-year 
or two-year Associate 
Degree of Nursing 
curriculums. Students 
also participated in 

an internship rotation 
throughout the 
intervention period. 


High-fidelity simulations 
were a relevant feature 
of this intervention. 


foxeyateliareyal 


Business-as- 
usual: 
Comparison 
students were 
offered the 
intervention, but 
self-selected into 
the comparison 
group. They 
had access 

to traditional 
clinical learning 
experiences. 


Size 
Technical skill 
proficiency: 

g = 0.36 
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Outcome Domain 


and WWC 


Calculated Effect 


Size’ 


Fein & Beecroft (2006) 
RCT 


Meets WWC group 
design standards 
without reservations 


Gardiner et al. (2017) 
RCT 


Meets WWC group 
design standards 
without reservations 


1,043 students 


age is not 
reported in the 
study 


1,217 students 


79% are 25 

years or older 

* 8.4% are 20 
or younger 

* 13.0% are 
21 to 24 

* 31.7% are 
25 to 34 

* 46.9% are 
35 or older 


1 community 
college 

and 1 local 
county 
social 
services 
department 
in California 


1 community 
college 

and local 
workforce 
agency in 
Arizona 


New Visions: Students 
participated in a 25- 
week training program 
that included basic 
skills remediation, 
occupational skills, 

life skills, and college 
prep courses. Students 
then participated in 
occupational training 
programs for at least 20 
hours a week. Students 
were also provided 
support services. 


Arelevant feature 

of this program 

was flexible course 
scheduling, including 
evening sessions, to 
accommodate students’ 
varying work, childcare, 
and transportation needs. 


Pathways to Healthcare: 
The intervention 


offered occupation 
training programs in the 
Healthcare Industry. 

A 10-week basic skills 
“bridge class” was 
offered to participants 
who needed to increase 
their skills prior to starting 
a training program. The 
intervention also offered 
advising, scholarships 
for tuition, books, and 
program supplies, job 
assistance services, and 
peer support groups. 


A relevant feature of this 
program was the option 
for students to choose 
the format of their basic 
skills program, either 
traditional format or 

a self-paced course, 
depending on their 
timeline and priorities. 


Business- 
as-usual: 
Students in the 
comparison 
group had 
access to 
courses and 


student support 
services at local 


educational 
institutions. 


Business-as- 
usual: The 
comparison 
group had 
access to 
services 
including 
standard 
assessment, 
basis skills 
education, 
occupational 
training 
programs, and 


support services 


(e.g., advising 
and tutoring), 


and employment 


services that 
are regularly 
available at 
the community 
college. 


Credit 
accumulation:° 


earned 1.5 more 
credits on average 
than business-as- 


usual did 


Postsecondary 


degree attainment: 


g = 0.28 


Credit 
accumulation: 
g = -0.07 


Industry- 
recognized 
credential 
attainment: 
g = 0.20* 
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COTVi cero) nat-m Blo) tat-lin 
and WWC 
Calculated Effect 
Size> 


Hamadyk & 
Zeidenberg (2018) 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
943 students 


69% are 25 

years or older 

° 14.3% 
are 20 or 
younger 

* 16.4% are 
21 to 24 

° 27.7% are 
25 to 34 

° 41.6% are 
35 or older 


Setting 

1 community 
college in 
lowa 


WTA Connect: 

Students enrolled in 
tuition-free remedial 

and occupational 
certificate courses in 
healthcare, advanced 
manufacturing, and 
administrative support. 
Math and Reading basic 
skills training was also 
offered. Support services 
included “achievement 
coaches” or advising 
support, transportation 
assistance, screening 
for public benefits, life 
skill workshops, and 
employment support 
services. 


A relevant feature of this 
program included self- 
paced, online basic skills 
remediation courses that 
students could choose 
to take in an instructor- 
supervised lab or on 
their own. 


Business- 
as-usual: 
Students in the 
comparison 
group had 
access to 
standard 
courses and 
student support 
services, 
including 
standard 

basic skills 
remediation 
software and 
retest system to 
reapply to Des 
Moines Area 
Community 
College. 


Industry- 
recognized 
credential 
attainment: 
g = 0.25* 
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(ole vere) pat-m Bleyaat-lia) 
and WWC 
Calculated Effect 


Participants’ | Setting ai teVareVaid(olamexelarelit(ola mmm exelarelid(eya) Size 
Martinson et al. (2018) 800 students Non- Carreras en Salud: Business-as- Industry- 
profit and Students enrolled in one — usual: recognized 
RCT 56% are 25 community | or more one-semester Students inthe — credential 
years or older college healthcare academies comparison attainment: 
Mog Te one ° 17.9% partnership that provided them with — group had g = 0.60* 
design standards arena : ‘ 
without reservations are 20 or in Illinois academic preparation, access to 
younger an accelerated path standard 
* 26.6% are through remediation in courses and 
21 to 24 occupational courses, services at 
° 34.1% are and integrated basic Instituto or other 
25 to 34 skills and remediation local community 
° 21.4% are courses. Support colleges or 
35 or older services included one- institutions, 
on-one advising, financial including 
support, childcare basic skill and 
and transportation occupational 
assistance, and training classes, 
employment support and standard 
services. financial aid 
assistance 
A relevant feature of this employment, 
program was evening and advising 
courses to accommodate services. 
students who were 
employed while attending 
the program. 
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COlVi cero) rat-m Ploy tat-lin 
and WWC 
Calculated Effect 
Size? 


Visher & Teres (2011) 
RCT 


Meets WWC group 
design standards 
without reservations 


917 students 


15% are 25 

years or older 

° 57.9% are 
18 to 20 

° 27.6% are 
21 to 25 

° 5.7% are 26 
to 30 

* 8.9% are 31 
or older 


1 community 
college in 
New York 


Career Focused 
Learning Communities: 
The intervention 
consisted of linked three 
courses that focused on 
project-based learning 
and interdisciplinary 
teaching. The goal was 
to expose students to 

a specific major area 

of study and career 
options. Instructors spent 
a significant amount 

of time planning their 
courses, and were given 
the support to do so. 


A relevant feature of 
this program was block 
scheduling, which 
reduced the amount 

of time students were 
required to be on 
campus. 


Business-as- 
usual: Students 
in the control 
group had 
access to other 
courses and 
student support 
services at the 
college. KCC 
offers a high- 
quality college 
experience. 


Credit 
accumulation: 
g=0.01 


Postsecondary 
degree attainment: 
g = 0.38* 


General academic 
achievement 
(college): 

g = 0.06 


Note: Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention ona 


distinct sample. 


@ Sample size represents the maximum number of participants in the study. In some studies, the number of participants varied 
across the outcome measures. 


> Effect sizes presented are from the outcome closest to the end of the intervention. For brevity, only the domain average effect 
size and statistical significance are reported in this table. 


° There was insufficient information reported in Fein & Beecroft (2006) to generate effect sizes in the Credit Accumulation 
and domain. In the table, coefficients from the analysis models are presented. The authors cited this domain as statistically 


significant. 


* = statistically significant at the .05 level. 
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Recommendation 4: Provide coordinated comprehensive student 
supports to improve credit accumulation and completion of non-degree 
credentials along career pathways. 


Rationale for Level of Evidence: Moderate 


WWC staff and the expert panel assigned Recommendation 4 a moderate level of evidence, based on 
15 studies of interventions that include comprehensive student supports. Of them, 12 studies? meet 
WWC group design standards without reservations, and the other three studies®° meet WWC group 
design standards with reservations. An additional three studies! include comprehensive student 
supports but does not report any educational outcomes. 


Across the 15 studies, there were findings in three outcome domains (Table C.5) even though all five 
educational outcome domains were relevant for this recommendation. Two domains had statistically 
significant, positive meta-analytic effect sizes: credit accumulation (g = 0.10, p < 0.01) and industry- 
recognized credential attainment (g = 0.48, p < 0.01). One domain had a statistically significant, 
negative meta-analytic effect size: postsecondary degree attainment (g = -0.12, p < 0.01). 


Table C.9. Domain-level effect sizes across the 15 studies supporting Recommendation 4 


Number bobs Oxo) ari(e(-¥ale=) 
Domain of studies Effect size alk=yave-1| p Value 
Credit accumulation 6 | 0.10 | 0.07, 0.12 | < 0.01 
Postsecondary degree attainment 6 | -0.12. —0.19, —0.06 | < 0.01 
Industry-recognized credential attainment | 14 0.48 0.45, 0.50 < 0.01 


Note: Significant findings are bolded. ns = nonsignificant findings. All effect sizes were calculated using a fixed effects 
meta-analytic effect size across studies. None of the 15 studies included findings that meet WWC standards in the following 
outcome domains: (1) academic performance; or (2) technical skill proficiency. 


Evidence from 14 of the 15 studies provides a direct test of the recommendation, as comprehensive 
student supports are a major component of the interventions. In Martinson et al. (2018), students 
received one-on-one advising, childcare and transportation assistance, tuition assistance, and referrals 
to other resources and services. In Rolston et al. (2017), the intervention required weekly counseling 
sessions and offered substantial financial support. In Martinson et al. (2016; Grand Rapids Community 
College and Kern Community College District), supportive services included financial assistance, 
tutoring, advising, and job search assistance. In Martinson et al. (2016; North Central Texas College), 
supportive services included financial assistance, tutoring, advising, and job search assistance; 
however, only financial assistance services were tested in the study. In Fein & Beecroft (2006), the 
intervention offered intensive advising and counseling as well as a supportive learning community. 

In Gardiner et al. (2017), students were offered intensive/proactive advising, financial assistance, 

and employment services. In Cook et al. (2018), the intervention provided advising supports, among 
other services. In Price et al. (2016), the intervention provided intensive academic supports as well 

as enhanced non-academic support services to addresses personal issues. In Price et al. (2017), 
colleges provided comprehensive support services that addressed career, personal, and academic 
issues. In Glosser et al. (2018), students received advising and had access to significant financial 
supports, tutoring and other academic supports, and emergency childcare and transportation. In 
Anderson et al. (2017), students were offered comprehensive academic supports including tutoring, 
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academic and career advising, and case management services. In Modicamore et al. (2017), a career 
navigator provided individual academic and career advising as well as assistance finding other 
supports. In Roder & Elliott (2018), the intervention centered on providing comprehensive support 
services including remediation, financial support, one-on-one counseling, life skills workshops, and 
employment supports. 


In one study (Hamadyk & Zeidenberg, 2018), there was limited uptake of the optional supportive 
services. 


The collection of studies demonstrate high external and moderate internal validity and a 
preponderance of positive impacts. In 14 of the 15 studies supporting this recommendation, 
comprehensive student supports are a major component of the tested intervention. Therefore, 
the expert panel has assigned a moderate level of evidence to this recommendation. This rating is 
supported by the strength of the evidence according to the following criteria: 


¢ Internal and External Validity of Supporting Evidence. The average effect sizes for two of the 
three educational outcome domains (credit accumulation and industry-recognized credential) are 
positive and statistically significant, and each outcome domain average is based on more than one 
study with a total sample size above 350. However, the average effect sizes for the positive outcome 
domains are predominantly derived from studies that meet WWC group design standards with 
reservations. 


¢ Consistency of Effects on Relevant Outcomes. The average effect size is statistically significant 
and negative for one of the three educational outcome domains (postsecondary degree attainment), 
but it is statistically significant and positive for the other two educational outcome domains (credit 
accumulation and industry-recognized credential). 


¢ Relationship Between the Evidence and Recommendation. In 14 of the 15 studies used to 
support Recommendation 4, comprehensive student supports are a major component of the 
tested intervention. 
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Table C.10. Studies providing evidence for Recommendation 4: Provide coordinated comprehensive 
student supports to improve credit accumulation and completion of non-degree credentials along 


career pathways 


Participants? 


Setting 


Talecdavceaiilelamexeyareliareyal 


Comparison 
Condition 


COlVi cero) sat-m Blo) nit-lin 
and WWC 
Calculated Effect 
Size 


Anderson et al. (2017) 


Additional source: 
Anderson et al. (2015) 


QED 


Meets WWC group 
design standards with 
reservations 


42,984 
students 


average age: 
28.3 years 


Community 
colleges 
and adult 
education 
settings in 
4 states (IL, 
KS, KY, LA) 


Accelerating Opportunity Business-as- 
(AQ): Community and usual: The 
technical colleges comparison 
that were in the AO group students 
intervention condition were drawn 


developed or modified 
existing programs that 
offered career pathways 
for in-demand jobs. 

A major component 

of AO was integrated 
instruction, where 
both basic skills and 
CTE instructors taught 
the same class with 

at least 25 percent 
overlap. Students also 
received additional 
services, including 
advising, navigational, 
and financial supports, 
to help them progress 
through the program. 


One relevant feature of 
this program was the use 
of career navigators to 
help students connect 
with student support 
services. 


from the same 
recruitment 
sources (adult 
education, 
developmental 
education, or 
CTE) as the 
intervention 
group, but they 
did not have 
the opportunity 
to participate in 
AO. 


Credit 
accumulation: 
g = -0.03 


Industry- 
recognized 
credential 
attainment: 
g = 0.49* 
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Outcome Domain 
and WWC 
Calculated Effect 


Participants’ | Setting Tai <eyaVeVaid(olam exe) atelit(ola Mmm exeyarelit(eya) Size’ 
Cook et al. (2018) 500 students 1 community MATC Patient Care Business-as- Credit 
college in Pathway: The Patient usual: The accumulation: 
RCT 55% are - Wisconsin Care Pathway comparison g=0.10 
years or olaer intervention adapted group 
Hee wale vee ° a and linked existing participants Industry- : 
eh Reet are <U or healthcare bridge had access to roenize 
younger programs into three one- usual services, Credential 
* 21.2% are semester Patient Care __ including attainment: 
21 to 24 Pathway academies that regular courses, 9 = ~0-08 
* 30.0% are combined occupational — advising 
25 to 34 training and basic skills services, and 
* 25.4% are remediation. Students other student 
35 or older were also provided with support services. 
dedicated advisors. 
Relevant features of 
this program included 
instructors and staff 
coordinating their 
student support at critical 
transition points in the 
career pathway, as well 
as following up with 
students after program 
completion. 
Fein & Beecroft (2006) 1,043 students 1 community New Visions: Students Business- Credit 
college participated in a 25- as-usual: accumulation:° 
RCT age is not and 1 local _—_ week training program Students inthe earned 1.5 more 
Meets WWC group reported in the county social that included basic comparison credits on average 
sestrn Senseo study services skills remediation, group had than business-as- 
: F department — occupational skills, access to usual did 
without reservations : ae ; : 
in California _ life skills, and college courses and 
prep courses. Students student support Postsecondary 
then participated in services atlocal degree attainment: 
occupational training educational g = 0.28 
programs for at least 20 _ institutions. 
hours a week. Students 
were also provided 
support services. 
WWC 2021007 Designing and Delivering Career Pathways at Community Colleges | Appendix C | 103 


Setting 


Talecdavccalitelamexeyareliareya) 


Comparison 
Condition 


Appendix C 


fOlVi cero) rat-m BLoynit-lin 
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Gardiner et al. (2017) 
RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
1,217 students 


79% are 25 

years or older 

* 8.4% are 20 
or younger 

* 13.0% are 
21 to 24 

° 31.7% are 
25 to 34 

* 46.9% are 
35 or older 


1 community 
college 

and local 
workforce 
agency in 
Arizona 


Pathways to Healthcare: 
The intervention 


offered occupation 
training programs in the 
Healthcare Industry. 

A 10-week basic skills 
“bridge class” was 
offered to participants 
who needed to increase 
their skills prior to starting 
a training program. The 
intervention also offered 
advising, scholarships 
for tuition, books, and 
program supplies, job 
assistance services, and 
peer support groups. 


Business-as- 
usual: The 
comparison 
group had 
access to 
services 
including 
standard 
assessment, 
basis skills 
education, 
occupational 
training 
programs, and 
support services 
(e.g., advising 
and tutoring), 
and employment 
services that 
are regularly 
available at 
the community 
college. 


Credit 
accumulation: 
g = -0.07 


Industry- 
recognized 
credential 
attainment: 
g = 0.20* 
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Outcome Domain 
and WWC 
Calculated Effect 
Size? 


Glosser et al. (2018) 
RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
631 students 


63% are 25 
years or older 


22.2% 

are 20 or 
younger 
14.9% are 
21 to 24 
29.8% are 
25 to 34 
33.2% are 
35 or older 


3 community 
colleges in 
Washington 


I-BEST: The I-BEST 
intervention includes 
courses that are part of 

a structured pathway, 
team-teaching between 
basic skills instructors 
and occupational 
instructors, and 
reimbursement to the 
college for 1.75 FTE per 
student. This intervention 
also included dedicated 
academic advisors/case 
managers, career/college 
navigation services, and 
financial assistance for 
training-related costs and 
transportation. 


Business-as- 
usual: 
Comparison 
group members 
could not access 
the I-BEST 
intervention and 
courses at the 
three colleges; 
however, they 
could access 
other education 
and training 
opportunities 
available to 
them, including 
non-l-BEST 
courses and 

the I-BEST 
intervention at 
other colleges. 
Both treatment 
and comparison 
group members 
could access 
general college 
advising, 
tutoring, financial 
aid services, and 
job assistance 
services that 
were available 
to all students at 
the colleges. 


Industry- 
recognized 
credential 
attainment: 
g = 0.49* 
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Outcome Domain 
Fe lavem Alf: 
Calculated Effect 


Participants’ | Setting TaiccYav(cValdcolamexeyacelit(olammimexeyarelii(oyal Size” 
Hamadyk & 943 students 1 community WTA Connect: Business-as- Industry- 
Zeidenberg (2018) college in Students enrolled in usual: recognized 
69% are 25 lowa tuition-free remedial Students inthe — credential 
RCT years or older and occupational comparison attainment: 
Meets WWC group * 14.3% certificate courses in group had g = 0.25* 
siesta Seandeae are 20 or healthcare, advanced access to 
WHEROuE ede niehone younger manufacturing, and standard 
* 16.4% are administrative support. courses and 
21 to 24 Math and Reading basic | student support 
° 27.7% are skills training was also services, 
25 to 34 offered. Support services including 
° 41.6% are included “achievement standard 
35 or older coaches” or advising basic skills 
support, transportation remediation 
assistance, screening software and 
for public benefits, life retest system to 
skill workshops, and reapply to Des 
employment support Moines Area 
services. Community 
College. 
Martinson et al. (2018) 800 students Non- Carreras en Salud: Business-as- Industry- 
profit and Students enrolled in one — usual: recognized 
RCT 56% are 25 community — or more one-semester Students inthe — credential 
ears or older college healthcare academies comparison attainment: 
Megs PO aroun, 17.9% partnership that provided them with — group had g = 0.60* 
design standards asi nS : . 
WithGueRaSamatons are 20 or in Illinois academic preparation, access to 
younger an accelerated path standard 
° 26.6% are through remediation in courses and 
21 to 24 occupational courses, services at 
° 34.1% are and integrated basic Instituto or other 
25 to 34 skills and remediation local community 
° 21.4% are courses. Support colleges or 
35 or older services included one- institutions, 
on-one advising, financial including 
support, childcare basic skill and 
and transportation occupational 
assistance, and training classes, 
employment support and standard 
services. financial aid 
assistance 
One relevant feature of employment, 
this program was the and advising 
wraparound student services. 
support services. 
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Comparison 
Condition 
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COlVi cero) rat-m BYoynit-lin 
and WWC 
Calculated Effect 
Size> 


Participants? 


Setting 


participants started with 
a six-week foundational 
training that integrated 
occupational training, 
basic skills, and 
employment support 
services. 


Martinson et al. (2016) 274 students 1 community GRCC Pathways to Business-as- Postsecondary 
college in Prosperity: All GJ-HC usual: Across degree attainment: 
Grand Rapids average age: Michigan interventions included community g = 0.45* 
Community College —_ 40.4 years a sequenced series college sites, 
RCT ° 5.4% are 21 of connected training comparison Industry- 
or younger courses linked to in- group members _ recognized 
Meets WWC group * 14.1% are demand occupations and could not access credential 
design standards 22 to 29 integrated occupational _ the intervention’s attainment: 
without reservations * 28.5% are skills and basic skills grant funded g = 1.00* 
30 to 39 instruction. Supportive services, but 
¢ 52.0% are services included could access 
40 or older financial assistance, other similar 
tutoring, advising, and services 
employment support available in the 
services. At GRCC, community. 
8-week training courses 
were offered in green- 
related sectors. Basic 
skills training was 
provided to those who 
needed them before 
starting training. 
Martinson et al. (2016) 816 students 1 community KCCD Clean Energy Business-as- Postsecondary 
college in Center: All GJU-HC usual: Across degree attainment: 
Kern Community average age: California interventions included community g = -2.24* 
College District 32.3 years a sequenced series college sites, 
° 19.2% of connected training comparison Industry- 
RCT ; : : 
are 21 or courses linked to in- group members _ recognized 
Meets WWC group younger demand occupations and could not access credential 
design standards * 30.8% are integrated occupational _ the intervention’s attainment: 
without reservations 22 to 29 skills and basic skills grant funded g = 1.39* 
° 24.8% are instruction. Supportive services, but 
30 to 39 services included could access 
° 25.2% are financial assistance, other similar 
40 or older tutoring, advising, and services 
employment support available in the 
services. At KCCD, all community. 
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Outcome Domain 
and WWC 
Calculated Effect 


Participants’ | Setting Tai <etaVeVaid(olamexeyatelit(ola Mmm exelarelit(eya) Size 
Martinson et al. (2016) 984 students 1 community NCTC Health Matrix Business-as- Postsecondary 
college in Grant scholarship usual: Across degree attainment: 
North Central Texas — average age: +Texas program: All GJ-HC community g = -0.08 
College 31.1 years interventions included college sites, 
* 28.6% a sequenced series comparison Industry- 
RCT sot : 
are 21 or of connected training group members _ recognized 
Meets WWC group younger courses linked to in- could not access _ credential 
design standards * 27.6% are demand occupations and _ the intervention’s attainment: 
without reservations 22 to 29 integrated occupational = grant funded g = 0.50* 
° 17.9% are skills and basic skills services, but 
30 to 39 instruction. Supportive could access 
° 25.8% are services included other similar 
40 or older financial assistance, services 
tutoring, advising, and available in the 
employment support community. 
services. NCTC focused 
on providing financial 
supports in the form 
of scholarships to its 
participants. 
Modicamore et al. 2,168 students 9 sites Accelerating Connections Business-as- Industry- 
(2017) across 4 to Employment (ACE): usual: The recognized 
average age: _ different The ACE provides comparison credential 
RCT 35.5 years states (CT, __ training that integrates condition was attainment: 
Meets WWC group ° 78% are GA, MD, TX) basic and vocational business as g = 0.45* 
design standards older than skills instruction. It also usual which 
ee en ee 25 years offers support services included access 
such as individual to any non-ACE 
coaching, tutoring, services at the 
financial assistance, and local workforce 
internship/job readiness agency. 
and placement services. 
Relevant features of 
this program included 
wraparound services 
for students, including 
both academic and 
non-academic supports, 
as well as career 
coordinators, supporting 
students with career 
planning and transition. 
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fOlVi cero) rat-m Yo nit-lin 
and WWC 
Calculated Effect 
Size? 


Price et al. (2017) 
QED 


Meets WWC group 
design standards with 
reservations 


Participants? 
1,396 students 


average age: 

317 years 

¢ 58% are 
25 years or 
older 


Setting 
Collaboration 
of community 
colleges, 
industry 
groups, and 
workforce 
agencies in 
4 states (CT, 
MA, NH, NY) 


Northeast Resiliency 
Consortium: The 
consortium created 44 
continuing education to 
credit pathway and 40 
credentials that stack 
to associate degrees. 
Enhanced support 
services included job 
search and placement 
assistance, counseling, 
tutoring, and connected 
referrals to other 
supports. Employer 
partnerships also 
provided work-based 
learning opportunities 
and internships. 


Relevant features of this 
program included career 
counselors delivering 
proactive, personalized 
one-on-one advising, 
integrating student 
supports in orientations 
and in-class activities, 
and facilitating weekly 
workshops on self- 

care and resiliency. 
Student success teams, 
consisting of instructors 
and student support 
staff, supported a more 
coordinated integrated 
delivery of student 
supports. 


Matched 
business-as- 
usual: The 
comparison 
group students 
were enrolled in 
similar programs 
of study offered 
by the technical 
colleges and 
had access to 
usual courses, 
programs, and 
student support 
services. 


Credit 
accumulation: 
g = 0.61* 


Industry- 
recognized 
credential 
attainment: 
g = 0.80* 
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fOlVi cero) rat-m Yo nit-lin 
and WWC 
Calculated Effect 
Size? 


Price et al. (2016) 
QED 


Meets WWC group 
design standards with 
reservations 


Participants? 
6,300 students 


average age: 
26.6 years 
for treatment 
group 


16 

community 
colleges in 
Wisconsin 


Making the Future: The 
intervention was meant 


to facilitate students’ 
ability to earn short- 
term credentials, which 
typically can be earned 
in less than one year, 
and “stack” toward 
one-year and two-year 
technical diplomas. The 
intervention included 
stacked and latticed 
manufacturing programs 
in welding, machine tool 
and computer numerical 
control (CNC -i.e., 

for 3D printing), and 
industrial maintenance. 
Support services varied 
depending on the college 
but included advising, 
counseling, and tutoring. 


One relevant feature 
of this program was 
the career counselors 
supporting students 
with career planning 
and transition to 
further education and 
employment. 


Business-as- 
usual: The 
comparison 
group students 
were enrolled in 
similar programs 
of study offered 
by the technical 
colleges and 
had access to 
typical courses, 
programs, and 
student support 
services. 


Credit 
accumulation: 
g = 0.19* 


Industry- 
recognized 
credential 
attainment: 
g = 0.35* 
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Calculated Effect 
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Roder & Elliott (2018) 


Additional source: 
Roder & Elliott (2019) 
Elliott & Roder (2017) 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
410 students 


71% are older 

than 25 years 

* 29.4% are 
18 to 24 

° 44.9% are 
25 to 34 

° 25.7% are 
35 to 64 


Setting 

2 community- 
based 
organizations 
in Texas 


Project QUEST: 
The intervention 


provides comprehensive 
support services to 
ensure individuals 
complete occupational 
training courses at 
local community 
colleges and obtain 
employment. These 
support services include 
basic skills instruction 
in math and reading, 
college navigation and 
enrollment assistance, 
financial assistance for 
training, certification 
and related expenses, 
transportation 
reimbursement, 
counseling, referrals 
to welfare agencies, 
required peer support 
and life skills trainings, 
and job readiness/ 
placement services. 


Relevant features of this 
program included holistic 
approach to advising 
(combining academic 
and non-academic 
student supports), 
mandatory weekly group 
meetings with advisors, 
and embedding course 
attendance audits in 
these. The program 
used braided funding, 
combining public and 
private sector funds, to 
cover the program costs. 


Business-as- 
usual: The 
comparison 
condition could 
not receive 
QUEST 
services, but 
could enroll in 
the college and 
training on their 
own and seek 
other services in 
the community. 


Postsecondary 
degree attainment: 
g = -0.29* 


Industry- 
recognized 
credential 
attainment: 
g = 0.61* 
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Outcome Domain 


and WWC 


Calculated Effect 


Size 


Rolston, Copson, & 
Gardiner (2017) 


RCT 


Meets WWC group 
design standards 
without reservations 


Note: Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention on a 


distinct sample. 


4 Sample size represents the maximum number of participants in the study. In some studies, the number of participants varied 


958 students 


63% are 25 

years or older 

° 14.1% 
are 20 or 
younger 

° 22.9% are 
21 to 24 

* 40.6% are 
25 to 34 

° 22.4% are 
35 or older 


across the outcome measures. 


> Effect sizes presented are from the outcome closest to the end of the intervention. For brevity, only the domain average effect 


1 community- 
based 
organization 
in Texas 


size and statistical significance are reported in this table. 


VIDA Program: The 
intervention required 


participants to enroll 
full-time in a certificate, 
associate’s, or 
baccalaureate program. 
A 16-week basic skills 
class was offered 

to participants who 
needed to increase their 
skills prior to starting 

a training program. 
Support services 
included required case 
management and 
counseling sessions, and 
financial assistance. 


Relevant features of 

this program included 

a holistic approach to 
advising (combining 
academic and non- 
academic student 
supports), weekly group 
meetings with advisors 
to support group learning 
and foster peer support 
networks, and weekly 
course attendance audits 
(embedded in group 
meetings). The program 
costs were covered 
using braided funding, 
combining public and 
private sector funds. 


Business-as- 
usual: Students 
had access to 
usual course and 
student support 
services at local 
educational 
institutions. 


Credit 


accumulation: 


g = 0.35* 


Postsecondary 
degree attainment: 


g = 0.14* 


Industry- 
recognized 
credential 
attainment: 
g = 0.18* 


° There was insufficient information reported in Fein & Beecroft (2006) to generate effect sizes in the Credit Accumulation 


and domain. In the table, coefficients from the analysis models are presented. The authors cited this domain as statistically 


significant. 


* = statistically significant at the .05 level. 
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Recommendation 5: Develop and continuously leverage partnerships to 
prepare students and advance their labor market success. 


Rationale for Level of Evidence: Moderate 


WWC staff and the expert panel assigned Recommendation 5 a moderate level of evidence, based 
on 11 studies of interventions that include employment-focused partnerships. Seven of the studies’ 
meet WWC group design standards without reservations, and the other four studies? meet WWC 
group design standards with reservations. An additional four studies include employment-focused 
partnerships but do not report any labor market outcomes. 


Across the 11 studies, there were findings in four outcome domains (Table C.3) even though all 

six labor market outcome domains were relevant for this recommendation. Three domains had 
statistically significant, positive meta-analytic effect sizes: short-term employment (g = 0.17, p < 0.01), 
short-term earnings (g = 0.33, p < 0.01), and medium-term earnings (g = 0.22, p < 0.05). The other 
domain (medium-term employment) did not have a statistically significant meta-analytic effect size. 


Table C.11. Domain-level effect sizes across the 11 studies supporting Recommendation 5 


Number Tob Oxe) ar i(e(-Yaler=) 
Domain of studies Effect size Talked ave-1| p Value 
Short-term employment | 8 0.17 | 0.13, 0.22 | < 0.01 
Medium-term employment 1 0.19 | —0.02, 0.40 | ns 
Short-term earnings 7 0.33 | 0.28, 0.37 | < 0.01 
Medium-term earnings 1 0.22 | 0.01, 0.43 < 0.05 


Note: Significant findings are bolded. ns = nonsignificant findings. All effect sizes were calculated using a fixed effects meta- 
analytic effect size across studies, except for the medium-term employment and medium-term earnings domains. These 
domains had findings from just one study; the effect sizes and p values presented here are the domain-level average effect 
sizes and p values for the individual relevant study. None of the 11 studies included findings that meet WWC standards in the 
following outcome domains: (1) long-term employment; or (2) long-term earnings. 


Evidence from 10 of the 11 studies provides an indirect test of the recommendation, as employment- 
focused partnerships are a secondary component of the interventions. In Martinson et al. (2018), 
healthcare employers and staffing agencies participated in an advisory council that informed the 
curriculum and helped set up clinical internships. In Hamadyk & Zeidenberg (2018), the career 
readiness and training courses embedded employer-related activities, including visits to and from 
employers. In Fein & Hamadyk (2018), employers were involved in the intervention in many ways 
including providing feedback on the curriculum, teaching workshops, participating or hosting 
events, and offering paid internships. In Martinson et al. (2016; Kern Community College District), 
staff cultivated and maintained relationships with employers to provide feedback on the curriculum, 
participate in events, and hire program completers on occasion. In Bragg et al. (2018), grant funds 
were used to enhance existing program advisory committees, which included employer participants 
who reviewed the curriculum. In Anderson et al. (2017), all sites partnered with local labor market 
partners, mainly Workforce Investment Boards (WIBs), to connect employers with students or to 
create work-based learning opportunities. In Modicamore et al. (2017), the intervention involved 
partnerships among WIBs, community colleges, and industry partners that provided employment 
to program graduates. In Roder & Elliott (2018), community-based organizations partnered with 
area colleges, training providers, and employers. In Dunham et al. (2016), college partnerships with 
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industry groups, employers, and policymakers were a central element of the intervention. In Price 
et al. (2018), existing partnerships with employers and local workforce groups were expanded to 
cover a wide array of activities such as recruitment, work-based learning opportunities, and 
curriculum development. 


One intervention was unable to continuously leverage partnerships. In Martinson et al. (2016, Grand 
Rapids Community College), employers were initially involved, but their role lessened over time. 


The collection of studies demonstrate high external and moderate internal validity and consistent 
positive impacts. In 10 of the 11 studies supporting this recommendation, employment-focused 
partnerships are a secondary component of the tested intervention. Therefore, the expert panel has 
assigned a moderate level of evidence to this recommendation. This rating is supported by the strength 
of the evidence according to the following criteria: 


¢ Internal and External Validity of Supporting Evidence. The average effect sizes for three of the 
four labor market outcome domains (short-term employment, short-term earnings, and medium- 
term earnings) are positive and statistically significant, and the outcome domain averages for short- 
term employment and short-term earnings are based on more than one study with a total sample 
size above 350. However, the average effect size for short-term employment is predominantly 
derived from studies that meet WWC group design standards with reservations, and the average 
effect size for medium-term earnings is based on one study. 


¢ Consistency of Effects on Relevant Outcomes. The average effect size is not statistically 
significant and negative for any of the four labor market outcome domains (short-term employment, 
medium-term employment, short-term earnings, or medium-term earnings). 


¢ Relationship Between the Evidence and Recommendation. In 10 of the 11 studies used to 
support Recommendation 5, employment-focused partnerships are a secondary component of the 
tested intervention. 
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Table C.12. Studies providing evidence for Recommendation 5: Develop and continuously leverage 


partnerships to prepare students and advance their labor market success 


Comparison 


(oye Kero) pat-m Bleyart-lin) 
Flare mm Ase: 
Calculated Effect 


Anderson et al. (2017) 


Additional source: 
Anderson et al. (2015) 


QED 


Meets WWC group 
design standards with 


Participants? 


42,984 
students 


average age: 
28.3 years 


Setting 
Community 
colleges 

and adult 
education 
settings in 4 
states (IL, KS, 
KY, LA) 


Taik=yac-Jaldcoyamexeyateliice)aMmimexeyalelit(oyal 
Accelerating Opportunity Business-as- 
(AQ): Community and usual: The 
technical colleges comparison 
that were in the AO group students 
intervention condition were drawn 


developed or modified 
existing programs that 
offered career pathways 


from the same 
recruitment 
sources (adult 


Size 
Short-term 


employment: 
g = 0.09* 


Short-term 
earnings:° Illinois 
participants 
earned $125 


reservations for in-demand jobs. education, OEE OH average 
A major component developmental iN a quarter than 
of AO was integrated education, or business-as-usual 
instruction, where CTE) as the did. Louisiana 
both basic skills and intervention participants 
CTE instructors taught group, but they earned $194 more 
the same class with did not have on average Ina 
at least 25 percent the opportunity quarter business- 
overlap. Students also to participate in @S-usual did. 
received additional AO. 
services, including 
advising, navigational, 
and financial supports, 
to help them progress 
through the program. 
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Bragg et al. (2018) 
QED 


Meets WWC group 
design standards with 
reservations 


Participants? 
1,448 
students 


average age: 
36 years 


77% are 25 
years or older 
° 4.9% are 21 
or younger 
18.2% are 
21 to 25 
19.5% are 
26 to 30 
25.6% are 
31 to 40 
17.7% are 
41 to 50 
13.8% are 
50 or older 


Setting 

1 community 
college in 
Utah 


Talecyavc-alite)amexeyateliareyal 


School of Applied 
Technology and 
Technical Studies 
(SATTS): The 
intervention converted 
24 certificate programs 
to competency based 
education (CBE) 

that were self-paced 
and self-directed. 

The intervention also 
enhanced student 
supports including 
academic advising and 
career/job assistance 
services. 


One relevant feature 
of this program was 

an advisory committee 
with employers and 
industry partners, 
engaged in part through 
partner roundtables, 

to guide the career 
pathway design and 
development. The 
program also shared 
data to help employers 
see the value of the 
program. 


Condition 
Matched 
business-as- 
usual: The 
comparison 
group students 
were enrolled in 
similar programs 
of study 
(technology) 
offered at the 
community 
college and 

had access to 
college-wide 
student support 
services. 


Size? 
Short-term 
employment: 
g = 0.20* 


Short-term 
earnings: 
g = 0.18* 
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Participants? | Setting TateiareVald(olamexelalelit(oyaMmmexeyareliicoyal Size” 
Dunham et al. (2016) 353 students 4 technical ShaleNET: The Workforce Short-term 
colleges in3 intervention is a assistance: employment: 
QED average age: states (PA, community based The comparison g = 0.38* 
35.2 years OH, TX) job training program group was 
cae Mitel ith 42.5% are that offered stackable drawn from the — Short-term 
ieee raiOnS older than credential training population of earnings: 
25 years programs in the oil and _ individuals from g = 6.46* 
gas industry. Post- the surrounding 
training internships counties who 
and other work- received 
based learning were assistance 
sometimes offered. with finding 
Support services employment 
included job search/ from federally 
placement services and funded 
career counseling. workforce 
programs in 
One relevant feature Pennsylvania 
of this program was an during the same 
advisory committee with time period. It 
employers and industry — excluded alll 
partners to guide the individuals who 
design and development received any 
of the career pathways. training. 
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Size? 


Fein & Hamadyk 
(2018) 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
2,544 
students 


° 57.2% are 
21 to 24 

° 42.8% are 
18 to 20 


Setting 

8 local 
offices that 
partner with 
community 
colleges 
(CA, DC, GA, 
IL, MA, NY, 
RI, WA) 


Year Up: The 
intervention provides six 
months of tuition-free 
full-time training in IT 
and financial services 
followed by six months 
of an internship. Young 
adults participated 

in a cohort-learning 
experience that integrate 
occupational, general, 
and contextualized basic 
skills contextualized. 
Support services 
included one-on-one 
advising, tutoring, 
community mentorship, 
case management, 
counseling, financial 
supports, and 
employment support 
services. 


Relevant features of this 
program included the 
strategic engagement 
of the advisory board to 
inform decision about 
the career pathways, 
using employer 
feedback for program 
improvement, facilitating 
work-based learning 
opportunities with 
employers, and a learn- 
and-earn model that 
allows for students to 
gain career-related work 
experiences. 


Business-as- 
usual: 
Comparison 
group members 
could receive 
other training, 
education, and 
supports in the 
community. 


Short-term 
earnings: 
g = 1.00* 
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Outcome Domain 


Flare mu a's {es 
Comparison Calculated Effect 
Participants? | Setting TateVareVard(olameceyatelitcolaMmmexeyarelii(eyal Size 
Hamadyk & 943 students 1community WTA Connect: Business- Short-term 
Zeidenberg (2018) college in Students enrolled in as-usual: earnings: 
69% are 25 lowa tuition-free remedial Students inthe g=0.03 
RCT years or older and occupational comparison 
Meets WWC group * 14.3% certificate courses in group had 
siesinn Seneae are 20 or healthcare, advanced access to 
WHOLE eee iehone younger manufacturing, and standard 
* 16.4% are administrative support. courses and 
21 to 24 Math and Reading student support 
* 27.7% are basic skills training services, 
25 to 34 was also offered. including 
¢ 41.6% are Support services standard 
35 or older included “achievement _ basic skills 
coaches” or advising remediation 
support, transportation software and 
assistance, screening retest system to 
for public benefits, life reapply to Des 
skill workshops, and Moines Area 
employment support Community 
services. College. 
Martinson et al. (2018) 800 students Non-profit and Carreras en Salud: Business-as- Short-term 
community Students enrolled in one usual: earnings: 
RCT 56% are 25 college or more one-semester Studentsinthe g=-0.04 
Meets WWC group years e older partnership in healthcare academies comparison 
design standards * 17.9% Illinois that provided them with group had 
WHRGIE SEH GHONS are 20 or academic preparation, access to 
younger an accelerated path standard 
* 26.6% are through remediation courses and 
21 to 24 in occupational services at 
¢ 34.1% are courses, and integrated _ Instituto or other 
25 to 34 basic skills and local community 
° 21.4% are remediation courses. colleges or 
35 or older Support services institutions, 
included one-on-one including 
advising, financial basic skill and 
support, childcare occupational 
and transportation training classes, 
assistance, and and standard 
employment support financial aid 
services. assistance 
employment, 
Relevant features of this ang advising 
program included an services. 
advisory committee with 
employers and industry 
partners, employers 
offering work-based 
learning opportunities, 
and using employer and 
student feedback for 
program improvement. 


WWC 2021007 Designing and Delivering Career Pathways at Community Colleges | Appendix C | 119 


Talecyavc-valile)amexeyaceliareyal 


Comparison 
Condition 


Appendix C 


Outcome Domain 
and WWC 
Calculated Effect 
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Martinson et al. (2016) 


Grand Rapids 
Community College 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
274 students 


average age: 

40.4 years 

° 5.4% are 21 
or younger 

° 14.1% are 
22 to 29 

* 28.5% are 
30 to 39 

¢ 52.0% are 
40 or older 


Setting 

1 community 
college in 
Michigan 


GRCC Pathways to 


Prosperity: All GJ- 
HC interventions 


included a sequenced 
series of connected 
training courses 

linked to in-demand 
occupations and 
integrated occupational 
skills and basic skills 
instruction. Supportive 
services included 
financial assistance, 
tutoring, advising, and 
employment support 
services. At GRCC, 
8-week training courses 
were offered in green- 
related sectors. Basic 
skills training was 
provided to those who 
needed them before 
starting training. 


Relevant features of 
this program included 
the career fairs and 
hiring events to facilitate 
student transition to 
employment. 


Business-as- 
usual: Across 
community 
college sites, 
comparison 
group members 
could not access 
the intervention’s 
grant funded 
services, but 
could access 
other similar 
services 
available in the 
community. 


Short-term 
employment: 
g = 0.18 
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Comparison 


Appendix C 


Outcome Domain 
and WWC 
Calculated Effect 


Participants? | Setting TateVareVald(olamexeyatelitcoyaMmmexeyareliicoyal Size’ 
Martinson et al. (2016) 816 students 1 community KCCD Clean Energy Business-as- Short-term 
college in Center: All GJ-HC usual: Across employment: 
Kern Community average age: == California interventions included = community g = 0.14 
College District 32.3 years a sequenced series college sites, 
RCT * 19.2% of connected training comparison 
are 21 or courses linked to in- group members 
Meets WWC group ; fet a demand occupations could not access 
design standards 99 to - and integrated — the intervention’s 
without reservations occupational skills grant funded 
* 24.8% are and basic skills services, but 
30 to 39 instruction. Supportive — could access 
* 25.2% are services included other similar 
40 or older financial assistance, services 
tutoring, advising, and _ available in the 
employment support community. 
services. At KCCD, all 
participants started with 
a six-week foundational 
training that integrated 
occupational training, 
basic skills, and 
employment support 
services. 
One relevant feature of 
this program was the 
strategic engagement 
of an advisory board 
with employers and 
industry partners. The 
program also included 
career fairs and hiring 
events to facilitate 
student transition to 
employment. 
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Talecyavc-valile)amexoyateliareyal 


Comparison 
Condition 


Appendix C 


Outcome Domain 
and WWC 
Calculated Effect 
Size? 


Modicamore et al. 
(2017) 


RCT 


Meets WWC group 
design standards 
without reservations 


Participants? 
2,168 
students 


average age: 

35.5 years 

* 78% are 
older than 
25 years 


Setting 

9 sites across 
4 different 
states (CT, 
GA, MD, TX) 


Accelerating | 
Connections to_ 


Employment (ACE): 
The ACE provides 
training that integrates 
basic and vocational 
skills instruction. It also 
offers support services 
such as individual 
coaching, tutoring, 
financial assistance, and 
internship/job readiness 
and placement services. 


Relevant features of 
this program included 
involving employers 
and industry partners 
in planning and design 
of the career pathways, 
using formal and 
informal feedback 

from students and 
employers for program 
improvement, facilitating 
work-based learning 
opportunities with 
employers, offering 
career fairs and 

hiring events, using a 
career navigator and 

a job developer to 
support post-program 
employment, and 
sharing data to help 
employers see the value 
of the program. 


Business-as- 
usual: The 
comparison 
condition was 
business as 
usual which 
included access 
to any non-ACE 
services at the 
local workforce 
agency. 


Short-term 
employment: 
g =0.27* 


Short-term 
earnings: 
g = 0.02 
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Comparison 
Condition 
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Outcome Domain 
and WWC 
Calculated Effect 
Size? 


Price et al. (2018) 
QED 


Meets WWC group 
design standards with 
reservations 


Participants? 
3,376 
students 


average age: 

28.5 years 

« 49% are 
under 25 

¢ 51% are 25 
or older 


14 community 
colleges in 
Wisconsin 


ACT for Healthcare: 
Students enrolled in 

a TAACCT funded- 
healthcare program 

and received at least 
one-grant funded 
support service. The 
support services offered 
at each college varied, 
but could include 
embedded in-class 
academic supports, 

out of class academic 
supports such as one 
to one tutoring, and 
non-academic supports 
such as life skills 
training, counseling, and 
employment support 
services. 


Matched 
business- 
as-usual: 
Students in the 
comparison 
group were 
matched to the 
intervention 
group and were 
enrolled in other 
healthcare 
programs at 

the consortium 
colleges that 
were not 
supported by the 
TAACCT grant 
during the grant 
period. Both 
the intervention 
and comparison 
groups could 
access other 
supportive 
services not 
funded through 
the grant. 


Short-term 
employment: 
g = 0.20* 
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Participants? 


Setting 
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Comparison 
Condition 
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Outcome Domain 
and WWC 
Calculated Effect 
Size? 


Roder & Elliott (2018) 


Additional source: 
Roder & Elliott (2019) 
Elliott & Roder (2017) 


RCT 


Meets WWC group 
design standards 
without reservations 


410 students 


71% are older 

than 25 years 

° 29.4% are 
18 to 24 

° 44.9% are 
25 to 34 

° 25.7% are 
35 to 64 


2 community- 
based 
organizations 
in Texas 


Project QUEST: The 
intervention provides 
comprehensive 
support services to 
ensure individuals 
complete occupational 
training courses at 
local community 
colleges and obtain 
employment. These 
support services include 
basic skills instruction 
in math and reading, 
college navigation and 
enrollment assistance, 
financial assistance for 
training, certification 
and related expenses, 
transportation 
reimbursement, 
counseling, referrals 
to welfare agencies, 
required peer support 
and life skills trainings, 
and job readiness/ 
placement services. 


Business-as- 
usual: The 
comparison 
condition could 
not receive 
QUEST 
services, but 
could enroll in 
the college and 
training on their 
own and seek 
other services in 
the community. 


Short-term 
employment: 
g=-0.17 


Short-term 
earnings: 
g=0.17 


Medium-term 
employment: 
g=0.19 


Medium-term 
earnings: 
g = 0.22* 


Note: Each row in this table represents a study, defined by the WWC as an examination of the effect of an intervention on a 


distinct sample. 


@ Sample size represents the maximum number of participants in the study. In some studies, the number of participants varied 
across the outcome measures. 


> Effect sizes presented are from the outcome closest to the end of the intervention. For brevity, only the domain average effect 
size and statistical significance are reported in this table. 


° There was insufficient information reported in Anderson et al. (2017) to generate effect sizes in the Short-term earnings 
domain. In the table, coefficients from the analysis models are presented. The authors cited Short-term earnings as statistically 


significant. 


* = statistically significant at the .05 level. 
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24; of them, 57 percent were age 21 or older. Two studies did not report the ages of students served. 
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professional-technical-education-instructional-design. 
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‘7 Wisconsin’s Nicolet Area Technical College offers a 16-credit short-term technical diploma in welding 
that recognizes a student completed the first semester of the one-year welding technical diploma 
program. Wisconsin Indianhead Technical College packaged five embedded technical diplomas 
within its one-year welding program. This allowed students the opportunity to learn one or more 
specific welding techniques that were identified as in demand by local employers; all these diplomas 
stack to a one-year welding technical diploma. 
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Strategies and Resources to Align Non-Credit and Credit Programs, available at http://edstrategy.org/ 
wp-content/uploads/2020/10/A-More-Unified-Community-College_FINAL.pdf. 


24 Martinson et al. (2018). 


> See https://facilitation.eku.edu/what-developing-curriculum-dacum for additional detail and 
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31 Fein & Beecroft (2006). 


> Roder & Elliott (2019) offer examples of how Project Quest offered students counseling to address 
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meetings that focus on life skills - such as time management, study skills, critical thinking, and 
conflict resolution - and coordinated referrals to multiple outside agencies for a host of supports. 


33 North Central Texas College (featured in Martinson et al., 2016) covered up to 95 percent of tuition 
for non-credit programs, the cost of course materials and certification exam fees, and child care 
reimbursements up to 70 percent. 


34 The University of Illinois’s Office of Community College Research and Leadership offers several 
resources on strategies to improve policies, programs, and practices at community colleges and to 
support students’ transition to college. See https://occr].illinois.edu/ for more information. 
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Modicamore et al. (2017). 
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media/documents/BT_toolkit_ June7.pdf); and the Texas Education Agency-funded manual 
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41 Price et al. (2018); Roder & Elliott (2018). 

” Cook et al. (2018). 

43 Gardiner et al. (2017). 

44 Martinson et al. (2018) and Price et al. (2016) offer examples. 

45 Anderson et al. (2015), Anderson et al. (2017), and Glosser et al. (2018) offer examples. 
46 Martinson et al. (2018). 

4” Anderson et al. (2015); Anderson et al. (2017). 

48 Glosser et al. (2018). 


49 See the Washington State Board for Community and Technical Colleges’s strategies, video and 
curricula samples for six team teaching models at: https://www.sbctc.edu/colleges-staff/ 
programs-services/i-best/team-teaching-models.aspx. The Breaking Through Contextualization 
Toolkit, which provides guidance, strategies, and tools for developing and implementing 
contextualized education, can be accessed here: https://jfforg-prod-new.s3.amazonaws.com/ 
media/documents/BT_toolkit_ June7.pdf 


°° Glosser et al. (2018, p. v). 
>! Modicamore et al. (2017). 
52 Glosser et al. (2018). 


°3 Glosser et al. (2018); Price et al. (2016). 
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>4 See https://www.jff.org/resources/accelerating-opportunity-braided-funding-toolkit/ for tips 
and strategies. 


55 See Recommendation 2 of the Using Technology to Support Postsecondary Student Learning 
practice guide for detailed suggestions (Dabbagh et al., 2019). 


°° Fein & Beecroft (2006); Gardiner et al. (2017); Hamadyk & Zeidenberg (2018); Martinson et 
al. (2018); Visher & Teres (2011). 


7 Curl et al. (2016). 
6 Bragg et al. (2018); Dunham et al. (2016); Price et al. (2018). 


°° Gardiner et al. (2017); Hamadyk & Zeidenberg (2018); Martinson et al. (2018); Visher & 
Teres (2011). 


6° Visher & Teres (2011). 

6! Martinson et al. (2018). 

® Fein & Beecroft (2006). 

63 Gardiner et al. (2017); Hamadyk & Zeidenberg (2018). 
64 For an example, see Cook et al. (2018). 


65 See Recommendation 2 of the Using Technology to Support Postsecondary Student Learning 
practice guide (Dabbagh et al., 2019). 


66 Bragg et al. (2018) offer an example. 
87 See https://www.educause.edu/research-and-publications/7-things-you-should-know-about. 


68 See https://library.educause.edu/topics/teaching-and-learning/online-course-development- 
planning and https://library.educause.edu/topics/teaching-and-learning/online-teaching- 
strategies. 


6° Cook et al. (2018). 

7 Roder & Elliott (2018) provide an example. 

1 Bragg et al. (2018); Price et al. (2016); Price et al. (2017). 
” Salt Lake Community College (2014). 


3 See https://www.cbenetwork.org/wp-content/uploads/2018/09/1st_button_CBE17016_Quality_ 
Framework_Update.pdf and https://www.cbenetwork.org/wp-content/uploads/2018/09/2nd_ 
button_Quality Framework_Users_ Guide Final_.pdf 


7 Dabbagh et al. (2019), p. 21. 


* For an example see Cook et al. (2018). 
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7 Curl et al. (2016). 
7 Roder & Elliott (2018). 


78 Cook et al. (2018); Fein & Beecroft (2006); Gardiner et al. (2017); Glosser et al. (2018); 
Hamadyk & Zeidenberg (2018); Martinson et al. (2016 - Grand Rapids Community College, 
Kern Community College District, North Central Texas College); Martinson et al. (2018); 
Modicamore et al. (2017); Roder & Elliott (2018); Rolston et al. (2017). 


79 Anderson et al. (2017); Price et al. (2016); Price et al. (2017). 
80 Dunham et al. (2016). 
8! Alliance for Quality Career Pathways (2014). 


82 Community colleges might also wish to engage in process mapping to map students’ journeys to 
and through a program and into employment. See https://er.educause.edu/articles/2018/3/using- 
process-mapping-to-redesign-the-student-experience for some resources for process mapping. 


83 See Martinson et al. (2018) and Modicamore et al. (2017) for examples. 
84 Elliott & Roder (2017); Fein & Hamadyk (2018); Modicamore et al. (2017); Rolston et al. (2017). 


85 See https://ccrc.tc.columbia.edu/media/k2/attachments/redesigning-community-college- 
onboarding-guided-pathways.pdf. For additional information about how guided pathways 
approaches have been used to help community college career-technical programs structure student 
supports, see Van Noy et al. (2016). 


86 Price et al. (2017). 

87 Cook et al. (2018). 

88 Cook et al. (2018). 

89 Roder & Elliott (2018); Rolston et al. (2017). 

°° See https://www.hobsons.com/solution/starfish/. 


°! Several career navigator toolkits are available to assist community colleges in planning and 
implementing career navigation supports. For example, see: http://www.collegetransition.org/ 
wp-content/uploads/2017/10/NavigatorManual-July2015.pdf; https://internal.cccs.edu/wp- 
content/uploads/documents/CCNIReferenceGuide.pdf; https://assets.aspeninstitute.org/ 
content/uploads/2017/05/Annotated-Bibliography-Of-Resources-For-Navigators.pdf; and 
https://denveroy.org/career-navigator-toolkit/. 


% Modicamore et al. (2017); Price et al. (2016). 
%3 Anderson et al. (2017). 

°4 Price et al. (2017). 

% Price et al. (2017). 


°6 Rolston et al. (2017). 
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°’ For an example see Bragg et al. (2018). 

%8 Price et al. (2017). 

°° Rolston et al. (2017). 

00 Roder & Elliott (2018). 

101 Price et al. (2017). 

‘022 Roder & Elliott (2018); Rolston et al. (2017). 
03 Roder & Elliott (2018). 


04 Fein & Hamadyk (2018); Hamadyk & Zeidenberg (2018); Martinson et al. (2016 - Grand 
Rapids Community College, Kern Community College District); Martinson et al. (2018); 
Modicamore et al. (2017); Roder & Elliott (2018). 


65 Anderson et al. (2017); Bragg et al. (2018); Dunham et al. (2016); Price et al. (2018). 
06 Gardiner et al. (2017); Price et al. (2016); Price et al. (2017); Rolston et al. (2017). 
107 Wilson (2015). 


168 Bragg et al. (2018); Dunham et al. (2016); Martinson et al. (2018); Price et al. (2016); 
Price et al. (2017). 


109 Fein & Hamadyk (2018); Martinson et al. (2016 - Kern Community College District, North 
Central Texas College). 


40 For an example see Bragg et al. (2018). 


1 See https://www.acteonline.org/professional-development/high-quality-cte-tools/business- 
and-community-partnerships/. 


2 Fein & Hamadyk (2018); Martinson et al. (2018); Modicamore et al. (2017); Price et al. (2016); 
Price et al. (2017). 


‘3 Gardiner et al. (2017); Martinson et al. (2016); Modicamore et al. (2017); Price et al. (2016); 
Price et al. (2017). 


"4 The Advancing Credentials initiative is led by the Center for Occupational Research and 
Development. The online Employer Engagement Toolkit also includes an employer engagement 
worksheet that a college can complete to track individual employers’ current involvements with it 
and to identify future partnership opportunities. See https://www.advancingcredentials.org/ 
toolkit/developing-your-program-college-activities.php. 


45 National Council for Workforce Education (n.d.) Activating Business and Industry Partners and 
Improving Advisory Committees. 


46 For an example see Modicamore et al. (2017). 
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«7 For more information on simulated work-based learning, see the National Center for Innovation in 
Career and Technical Education’s strategies in the following resource: https://www.gfcmsu.edu/ 
revup/documents/SWBL_Report.pdf. 


"8 McFarland et al. (2019). 
9 Gardner & Bartkus (2010). 
20 Fein & Hamadyk (2018). 


"1 See Gardiner et al. (2017) for a discussion of challenges that may emerge when re-engaging 
students after they exit a career pathway. 


2 Fein & Hamadyk (2018); Martinson et al. (2018). 

3 Martinson et al. (2018); Modicamore et al. (2017). 

24 See Bragg et al. (2018) for more information. 

5 Modicamore et al. (2017). 

26 See https://cte.ed.gov/wbltoolkit/engaging. html. 

"7 Fein & Hamadyk (2018); Gardiner et al. (2017). 

°8 Bragg et al. (2018); Modicamore et al. (2017); Price et al. (2016). 


9 See Tool 1.7 in Job for the Future’s Employer Engagement Toolkit for additional detail: https://jfforg- 
prod-new.s3.amazonaws.com/media/documents/Employer-Engagement-Toolkit-092315.pdf. 


0 Study reviews occurred using the WWC Version 4.0 Standards Handbook, available at https://ies. 
ed.gov/ncee/wwc/Handbooks, and the version 4.0 practice guide review protocol available at 
https://ies.ed.gov/ncee/wwc/Document/1294. 


31 Following WWC guidelines, improved outcomes are indicated by a positive, statistically significant 
effect from a meta-analytic effect size calculated separately for each relevant outcome domain. 
For more information on how the WWC calculates these effect sizes, see the WWC Version 4.1 
Procedures Handbook at https://ies.ed.gov/ncee/wwc/Handbooks. 


‘82 For more information, see the WWC Frequently Asked Questions page at https://ies.ed.gov/ncee/ 
wwc/FAQ. 


‘33 These findings are characterized as positive effects, potentially positive effects, uncertain effects, 
potentially negative effects, and negative effects. See Table IV.3 from the WWC Version 4.1 
Procedures Handbook. 


54 See Table IV.4 from the WWC Version 4.1 Procedures Handbook. 


85 CTE interventions designed to support progression toward or completion of a graduate degree fall 
outside of the scope of these reviews. The reason is that graduate work will generally entail highly 
specialized training delivered to students who likely already have base credentials needed for 
middle-class employment and wages. 
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36 Tf study interventions operate in multiple types of settings, at least 50% of the sample must be 
enrolled in community colleges for the study to be eligible. Alternatively, the study must include 
subgroup findings that focus on students enrolled in the community college sites. 


‘37 https://ies.ed.gov/ncee/wwc/Document/1294 


88 The updated What Works Clearinghouse Standards and Procedures Handbook, Version 4.1 was 
publically available at the time of release of this practice guide. However, the reviews of studies 
that contributed to this practice guide’s recommendations were conducted prior to that update. 
Therefore, the reviews for this practice guide were conducted under Version 4.0. Levels of evidence 
were assessed using a fixed-effects meta-analysis, following guidance in Version 4.1, however. 


89 The WWC chose the fixed-effects model because its goal is to make inferences about the studies in 
WWC intervention reports and practice guides. Unlike the fixed-effect (singular) model, the fixed- 
effects (plural) model does not assume that the studies are estimating a common effect. Instead, 
the fixed-effects model assumes that the observed variation among the effect sizes in the meta- 
analysis reflects the true variation in population effects. Accordingly, inferences to larger study 
populations are constrained to those that share the same patterns of important study characteristics 
that are related to effect size. See Hedges & Vevea (1998), What Works Clearinghouse Standards and 
Procedures Handbook, Version 4.1. 


The WWC carefully weighed the various options for meta-analysis, including using random effects 
estimation. We determined that the fixed-effects modeling approach is currently the best option 

for the WWC, based on the available research. Implementing a random effects model requires 
estimating the between-studies variance component, and this estimate is poor unless the meta- 
analysis includes a relatively large number of studies. The fixed-effects approach allows the WWC to 
conceptually acknowledge variability in effects. As of January 2020, the largest synthesis conducted 
by the WWC was based on nine studies, and most WWC intervention reports are far smaller: two or 
three studies is typical. Therefore, we believe that for the foreseeable future, it is not practical for 
the WWC to adopt a random effects model for its syntheses. https://ies.ed.gov/ncee/wwc/Docs/ 
referenceresources/SumResponsePublicComments-v4-1-508.pdf. 


40 Tf multiple contrasts from a study are entered into a meta-analysis, participants from experimental 
conditions that are common across contrasts will be counted twice, resulting in effect sizes that 
are statistically dependent. This dependence in a meta-analysis can create a serious threat to the 
validity of the results. 


41 For the labor market outcomes in the employment and earnings domains, the review considers 
short-, medium-, and long-term impacts as separate outcome domains. This yields a total of six 
domains for the Labor Market category. Short-term follow-up is defined as one to two years after 
earliest possible program completion; medium-term follow-up is defined as three to four years after 
earliest possible program completion; and long-term follow-up is defined as five or more years after 
earliest possible program completion. When a given labor market outcome is measured at different 
time points within the same outcome domain, the review prioritizes the measure with the shortest 
follow-up period. 


42 Fein & Hamadyk (2018); Gardiner et al. (2017); Glosser et al. (2018); Hamadyk & Zeidenberg 
(2018); Martinson et al. (2016 - Grand Rapids Community College, Kern Community College 
District); Martinson et al. (2018); Modicamore et al. (2017). 
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48 Anderson et al. (2017); Dunham et al. (2016); Price et al. (2016); Price et al. (2018). 


44 Cook et al. (2018); Gardiner et al. (2017); Glosser et al. (2018); Hamadyk & Zeidenberg 
(2018); Martinson et al. (2016 - Kern Community College District); Martinson et al. (2018); 
Modicamore et al. (2017). 


45 Anderson et al. (2017); Price et al. (2016). 


46 Fein & Beecroft (2006); Gardiner et al. (2017); Hamadyk & Zeidenberg (2018); Martinson et 
al. (2018); Visher & Teres (2011). 


47 Curl et al. (2016). 
48 Bragg et al. (2018); Dunham et al. (2016); Fein & Hamadyk (2018); Price et al. (2018). 


49 Cook et al. (2018); Fein & Beecroft (2006); Gardiner et al. (2017); Glosser et al. (2018); 
Hamadyk & Zeidenberg (2018); Martinson et al. (2016 - Grand Rapids Community College, 
Kern Community College District, North Central Texas College); Martinson et al. (2018); 
Modicamore et al. (2017); Roder & Elliott (2018); Rolston et al. (2017). 


‘50 Anderson et al. (2017); Price et al. (2016); Price et al. (2017). 
‘5! Dunham et al. (2016); Fein & Hamadyk (2018); Price et al. (2018). 


‘2 Fein & Hamadyk (2018); Hamadyk & Zeidenberg (2018); Martinson et al. (2016 - Grand 
Rapids Community College, Kern Community College District); Martinson et al. (2018); 
Modicamore et al. (2017); Roder & Elliott (2018). 


‘53 Anderson et al. (2017); Bragg et al. (2018); Dunham et al. (2016); Price et al. (2018). 
54 Gardiner et al. (2017); Price et al. (2016); Price et al. (2017); Rolston et al. (2017). 


5 Cook et al. (2018); Fein & Beecroft (2006); Fein & Hamadyk (2018); Gardiner et al. (2017); 
Glosser et al. (2018); Hamadyk & Zeidenberg (2018); Martinson et al. (2016 - Grand Rapids 
Community College, Kern Community College District, North Central Texas College); 
Martinson et al. (2018); Rolston et al. (2017). 


WWC 2021007 Designing and Delivering Career Pathways at Community Colleges | Notes | 143 


